Three-dimensional data generation method, three-dimensional data generation device, and recording medium

ABSTRACT

A three-dimensional data generation method includes the following processing. A processor acquires two or more images of a component inside a turbine. The processor detects two or more correspondence regions that are the same regions in at least two images. The processor determines whether at least part of a region of each image is a change region or a non-change region. The processor generates three-dimensional data by using a correspondence region determined to be the change region without using a correspondence region determined to be the non-change region.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a three-dimensional data generationmethod, a three-dimensional data generation device, and a recordingmedium for generating three-dimensional data indicating athree-dimensional shape inside a turbine.

Priority is claimed on Japanese Patent Application No. 2022-048909,filed on Mar. 24, 2022, the content of which is incorporated herein byreference.

Description of Related Art

Industrial endoscope devices have been used for inspection ofabnormalities (damage, corrosion, and the like) occurring insideindustrial equipment such as boilers, turbines, engines, pipes, and thelike. Various subjects are targets for inspection using industrialendoscope devices. Turbines used for aircraft and power generationfacilities are especially important subjects in the inspection usingindustrial endoscope devices.

In general, an industrial endoscope device uses a single-eye opticaladaptor and an optical adaptor used for measurement. The single-eyeoptical adaptor is used for normal observation of a subject. The opticaladaptor used for measurement is used for restoring three-dimensional(3D) information of a subject. For example, the optical adaptor used formeasurement is a stereo optical adaptor having two visual fields. Anindustrial endoscope device can measure the size of an abnormality thathas been found by using the 3D information. A user can check a shape(unevenness or the like) of the subject restored by using the 3Dinformation. The industrial endoscope device can record proof that aninspection has been performed by restoring a wide range of the structureof the subject that is an inspection target as the 3D information. Asdescribed above, the 3D information contributes to improving the qualityof the inspection and streamlining the inspection.

In recent years, techniques have been developed that restore a widerange of a structure of a subject as 3D information. For example,techniques have been developed that acquire an image of a subject byusing a single-eye optical adaptor and restore 3D information of thesubject by using the image. Such a technique executes 3D restorationprocessing based on a change of relative movement of a distal endportion of an endoscope to the subject and restores the 3D information.In addition, optical adaptors used for measurement that have stereooptical systems including devised optical systems have been developed.Such an optical adaptor used for measurement has an angle of view closeto that of a single-eye optical adaptor. By using the optical adaptorused for measurement and by applying the above-described techniques, anindustrial endoscope device can restore a wide range of a structure of asubject as 3D information.

Turbines are used for aircraft engines or power generators. Rotor bladesof the turbines are major subjects in inspection using an industrialendoscope device. One or more steps, each of which includes rotatablerotor blades and fixed stator blades, are disposed along a rotation axisinside a turbine in each of a compressor section and a turbine section.A shroud is disposed outside the rotor blades.

In general, abnormalities on rotor blades are searched for when therotor blades are rotating in an inspection of the rotor blades. In suchan inspection, stator blades or a shroud as well as the rotor bladesoften conic into a visual field of an endoscope. The stator blades andthe shroud do not move differently from the rotor blades. In a case inwhich the stator blades or the shroud along with the rotor blades areseen in an image acquired by using a single-eye optical adaptor or astereo optical adaptor, an industrial endoscope device may fail torestore 3D information of a subject.

Japanese Unexamined Patent Application, First Publication No.2020-126432 discloses a technique of restoring 3D information of asubject by using an image in which a moving object and a stationaryobject are seen in order to resolve the above-described problem. Thetechnique uses an image acquired by a camera installed in a running carso as to restore 3D information of a subject around the car.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention, athree-dimensional data generation method of generating three-dimensionaldata indicating a three-dimensional shape inside a turbine by using aprocessor is provided. The method includes acquiring two or more imagesof a component inside the turbine. The component includes a first objectcapable of moving inside the turbine and a second object that isstationary inside the turbine. The two or more images are generated byan imaging apparatus including a tubular insertion unit that acquires anoptical image inside the turbine. The insertion unit is inserted intothe turbine through a hole formed in the turbine. A moving direction ofthe insertion unit when the insertion unit is inserted into the turbineis different from a moving direction of the first object. A relativeposition of the first object to the insertion unit is different betweentimings at which the imaging apparatus generates images while the firstobject moves. The method includes detecting two or more correspondenceregions that are the same regions of the component in at least twoimages included in the two or more images. The method includesdetermining whether at least part of a region of each of the two or moreimages is a change region or a non-change region. The change region is aregion of the component of which coordinates in an image generated bythe imaging apparatus change. The non-change region is a region of thecomponent of which coordinates in an image generated by the imagingapparatus do not change. The method includes generating thethree-dimensional data by using a correspondence region determined to bethe change region among the two or more correspondence regions withoutusing a correspondence region determined to be the non-change regionamong the two or more correspondence regions.

According to a second aspect of the present invention, in the firstaspect, after the two or more correspondence regions are detected,determination as to whether the at least part of the region is thechange region or the non-change region may be executed.

According to a third aspect of the present invention, in the firstaspect, after determination as to whether the at least part of theregion is the change region or the non-change region is executed, thetwo or more correspondence regions may be detected by using the changeregion without using the non-change region.

According to a fourth aspect of the present invention, in the firstaspect, the first object may include a rotor blade. The second objectmay include a stator blade or a shroud.

According to a fifth aspect of the present invention, in the firstaspect, part of the first object may be concealed by the second objectin the two or more images.

According to a sixth aspect of the present invention, in the fifthaspect, the second object may include an object that conceals part ofthe first object in the two or more images. The second object mayinclude an object, part of which is concealed by the first object in thetwo or more images.

According to a seventh aspect of the present invention, in the firstaspect, the imaging apparatus may generate the two or more images at twoor more different timings. Determination as to whether the at least partof the region is the change region or the non-change region may beexecuted based on a moving amount of each of the correspondence regionsbetween at least two images included in the two or more images.

According to an eighth aspect of the present invention, in the firstaspect, determination as to whether the at least part of the region isthe change region or the non-change region may be executed based on adifference of a pixel value between at least two images included in thetwo or more images.

According to a ninth aspect of the present invention, in the firstaspect, determination as to whether the at least part of the region isthe change region or the non-change region may be executed bydetermining a subject seen in one image included in the two or moreimages.

According to a tenth aspect of the present invention, in the firstaspect, the method may include determining a position of the insertionunit inside the turbine. The method may include executing processing ofnotifying a user of information indicating a change of the position ofthe insertion unit when the position has changed.

According to an eleventh aspect of the present invention, in the firstaspect, the method may include calculating an area of the non-changeregion in an image included in the two or more images before generatingthe three-dimensional data. The method may include executing processingof notifying a user of an alert when the area is larger than apredetermined value.

According to a twelfth aspect of the present invention, in the firstaspect, the first object may rotate inside the turbine due to a drivingforce generated by a driving device. The method may include determiningwhether the first object has rotated once inside the turbine. The methodmay include executing processing of notifying a user of informationindicating that the first object has rotated once inside the turbinewhen it is determined that the first object has rotated once inside theturbine.

According to a thirteenth aspect of the present invention, in the firstaspect, the first object may rotate inside the turbine due to a drivingforce generated by a driving device. Detection of the two or morecorrespondence regions, determination as to whether the at least part ofthe region is the change region or the non-change region, and generationof the three-dimensional data may be repeatedly executed while the firstobject rotates. The generation of the three-dimensional data may bestopped and the detection of the two or more correspondence regions andthe determination as to whether the at least part of the region is thechange region or the non-change region may be continued when therotation of the first object has been stopped.

According to a fourteenth aspect of the present invention, in thethirteenth aspect, the generation of the three-dimensional data may berestarted when the first object starts to rotate again.

According to a fifteenth aspect of the present invention, in thethirteenth aspect, the method may include determining a rotation stateof the first object by using an image included in the two or moreimages.

According to a sixteenth aspect of the present invention, in thethirteenth aspect, the method may include determining a rotation stateof the first object by monitoring a state of the driving device.

According to a seventeenth aspect of the present invention, in the firstaspect, the insertion unit may be fixed inside the turbine.

According to an eighteenth aspect of the present invention, in the firstaspect, the imaging apparatus may be a borescope.

According to a nineteenth aspect of the present invention, athree-dimensional data generation system that generatesthree-dimensional data indicating a three-dimensional shape inside aturbine is provided. The three-dimensional data generation systemincludes an imaging apparatus including a tubular insertion unit thatacquires an optical image inside the turbine. The imaging apparatus isconfigured to generate two or more images of a component inside theturbine. The component includes a first object capable of moving insidethe turbine and a second object that is stationary inside the turbine.The insertion unit is inserted into the turbine through a hole formed inthe turbine. A moving direction of the insertion unit when the insertionunit is inserted into the turbine is different from a moving directionof the first object. A relative position of the first object to theinsertion unit is different between timings at which the imagingapparatus generates images while the first object moves. Thethree-dimensional data generation system includes a three-dimensionaldata generation device including a processor. The processor isconfigured to acquire the two or more images. The processor isconfigured to detect two or more correspondence regions that are thesame regions of the component in at least two images included in the twoor more images. The processor is configured to determine whether atleast part of a region of each of the two or more images is a changeregion or a non-change region. The change region is a region of thecomponent of which coordinates in an image generated by the imagingapparatus change. The non-change region is a region of the component ofwhich coordinates in an image generated by the imaging apparatus do notchange. The processor is configured to generate the three-dimensionaldata by using a correspondence region determined to be the change regionamong the two or more correspondence regions without using acorrespondence region determined to be the non-change region among thetwo or more correspondence regions.

According to a twentieth aspect of the present invention, in thenineteenth aspect, the imaging apparatus and the three-dimensional datageneration device may be included in an endoscope device.

According to a twenty-first aspect of the present invention, in thenineteenth aspect, the imaging apparatus may be included in an endoscopedevice. The three-dimensional data generation device may be included inan external device other than the endoscope device.

According to a twenty-second aspect of the present invention, anon-transitory computer-readable recording medium stores a programcausing a computer to execute processing of generating three-dimensionaldata indicating a three-dimensional shape inside a turbine. Theprocessing includes acquiring two or more images of a component insidethe turbine. The component includes a first object capable of movinginside the turbine and a second object that is stationary inside theturbine. The two or more images are generated by an imaging apparatusincluding a tubular insertion unit that acquires an optical image insidethe turbine. The insertion unit is inserted into the turbine through ahole formed in the turbine. A moving direction of the insertion unitwhen the insertion unit is inserted into the turbine is different from amoving direction of the first object. A relative position of the firstobject to the insertion unit is different between timings at which theimaging apparatus generates images while the first object moves. Theprocessing includes detecting two or more correspondence regions thatare the same regions of the component in at least two images included inthe two or more images. The processing includes determining whether atleast part of a region of each of the two or more images is a changeregion or a non-change region. The change region is a region of thecomponent of which coordinates in an image generated by the imagingapparatus change. The non-change region is a region of the component ofwhich coordinates in an image generated by the imaging apparatus do notchange. The processing includes generating the three-dimensional data byusing a correspondence region determined to be the change region amongthe two or more correspondence regions without using a correspondenceregion determined to be the non-change region among the two or morecorrespondence regions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view showing an entire configuration of anendoscope device according to a first embodiment of the presentinvention.

FIG. 2 is a block diagram showing an internal configuration of theendoscope device according to the first embodiment of the presentinvention.

FIG. 3 is a diagram schematically showing the disposition of rotorblades and stator blades inside a turbine in the first embodiment of thepresent invention.

FIG. 4 is a diagram schematically showing the disposition of the rotorblades inside the turbine in the first embodiment of the presentinvention.

FIG. 5 is a diagram showing an example of an image acquired by theendoscope device according to the first embodiment of the presentinvention.

FIG. 6 is a block diagram showing a functional configuration of a CPUincluded in the endoscope device according to the first embodiment ofthe present invention.

FIG. 7 is a flow chart showing a procedure of three-dimensional (3D)data generation processing in the first embodiment of the presentinvention.

FIG. 8 is a diagram showing an example of a feature region in the firstembodiment of the present invention.

FIG. 9 is a diagram showing an example of the same feature regionbetween two images in the first embodiment of the present invention.

FIG. 10 is a diagram showing a method of calculating a moving amount inthe first embodiment of the present invention.

FIG. 11 is a diagram showing a method of determining whether a region ofan image in the first embodiment of the present invention is a movingregion or a stationary region.

FIG. 12 is a schematic diagram showing a situation in which an image isacquired in the first embodiment of the present invention.

FIG. 13 is a flow chart showing a procedure of 3D restoration processingin the first embodiment of the present invention.

FIG. 14 is a diagram showing an example of information displayed on adisplay unit included in the endoscope device according to the firstembodiment of the present invention.

FIG. 15 is a diagram showing an example of information displayed on thedisplay unit included in the endoscope device according to the firstembodiment of the present invention.

FIG. 16 is a flow chart showing a procedure of 3D data generationprocessing in a first modified example of the first embodiment of thepresent invention.

FIG. 17 is a diagram showing a method of determining whether a region ofan image in the first modified example of the first embodiment of thepresent invention is a moving region or a stationary region.

FIG. 18 is a flow chart showing a procedure of 3D data generationprocessing in the first modified example of the first embodiment of thepresent invention.

FIG. 19 is a flow chart showing a procedure of 3D data generationprocessing in a second modified example of the first embodiment of thepresent invention.

FIG. 20A is a diagram showing an example of an image in the secondmodified example of the first embodiment of the present invention.

FIG. 20B is a diagram showing an example of a moving region and astationary region in the second modified example of the first embodimentof the present invention.

FIG. 21 is a diagram showing an example of object information in thesecond modified example of the first embodiment of the presentinvention.

FIG. 22 is a flow chart showing a procedure of processing executed by anendoscope device according to the second modified example of the firstembodiment of the present invention.

FIG. 23 is a block diagram showing a functional configuration of a CPUincluded in an endoscope device according to a second embodiment of thepresent invention.

FIG. 24 is a flow chart showing a procedure of 3D data generationprocessing in the second embodiment of the present invention.

FIG. 25 is a flow chart showing a procedure of 3D data generationprocessing in the second embodiment of the present invention.

FIG. 26A is a diagram showing an example of an image in the secondembodiment of the present invention.

FIG. 26B is a diagram showing an example of processing of determiningthe size of a stationary region in the second embodiment of the presentinvention.

FIG. 27A is a diagram showing an observation position of rotor blades inthe second embodiment of the present invention.

FIG. 27B is a diagram showing an observation position of rotor blades inthe second embodiment of the present invention.

FIG. 28A is a diagram showing an observation position of the rotorblades in the second embodiment of the present invention.

FIG. 28B is a diagram showing an observation position of the rotorblades in the second embodiment of the present invention.

FIG. 29 is a flow chart showing a procedure of 3D data generationprocessing in the second embodiment of the present invention.

FIG. 30 is a flow chart showing a procedure of 3D data generationprocessing in the second embodiment of the present invention.

FIG. 31 is a flow chart showing a procedure of 3D data generationprocessing in the second embodiment of the present invention.

FIG. 32 is a perspective view of a distal end of an insertion unit and astereo optical adaptor in an endoscope device according to a thirdembodiment of the present invention.

FIG. 33 is a cross-sectional view of the distal end of the insertionunit and the stereo optical adaptor in the endoscope device according tothe third embodiment of the present invention.

FIG. 34 is a flow chart showing a procedure of 3D data generationprocessing in the third embodiment of the present invention.

FIG. 35 is a diagram showing a method of calculating three-dimensionalcoordinates of a point of interest in the third embodiment of thepresent invention.

FIG. 36 is a block diagram showing a configuration of a distal end of aninsertion unit and a stereo optical adaptor in an endoscope deviceaccording to a fourth embodiment of the present invention.

FIG. 37 is a flow chart showing a procedure of 3D data generationprocessing in the fourth embodiment of the present invention.

FIG. 38 is a diagram showing a stereo image generated in the fourthembodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the drawings.

First Embodiment

A first embodiment of the present invention will be described.Hereinafter, an example in which a three-dimensional (3D) datageneration device is included in an endoscope device will be described.

A configuration of an endoscope device 1 in the first embodiment will bedescribed by using FIG. 1 and FIG. 2 . FIG. 1 shows an externalappearance of the endoscope device 1. FIG. 2 shows an internalconfiguration of the endoscope device 1.

The endoscope device 1 shown in FIG. 1 includes an insertion unit 2, amain body unit 3, an operation unit 4, and a display unit 5. Theendoscope device 1 images a subject and generates an image. The subjectis an industrial product. In order to observe various subjects, a usercan replace an optical adaptor mounted at a distal end 20 of theinsertion unit 2, select a built-in video-processing program, and add avideo-processing program.

The insertion unit 2 is inserted inside a subject. The insertion unit 2has a long and thin bendable tube shape from the distal end 20 to a baseend portion. The insertion unit 2 images a subject and outputs animaging signal to the main body unit 3. An optical adapter is mounted onthe distal end 20 of the insertion unit 2. For example, a single-eyeoptical adapter is mounted on the distal end 20. The main body unit 3 isa control device including a housing unit that houses the insertion unit2. The operation unit 4 accepts an operation for the endoscope device 1from a user. The display unit 5 includes a display screen and displaysan image of a subject acquired by the insertion unit 2, an operationmenu, and the like on the display screen.

The operation unit 4 is a user interface. The display unit 5 is amonitor (display) such as a liquid crystal display (LCD). The displayunit 5 may be a touch panel. In such a case, the operation unit 4 andthe display unit 5 are integrated.

The main body unit 3 shown in FIG. 2 includes an endoscope unit 8, acamera control unit (CCU) 9, and a control device 10.

The endoscope unit 8 includes a light source device and a bending devicenot shown in the drawing. The light source device provides the distalend 20 with illumination light that is necessary for observation. Thebending device bends a bending mechanism that is built in the insertionunit 2.

A lens 21 and an imaging device 28 are built in the distal end 20 of theinsertion unit 2. The lens 21 is an observation optical system. The lens21 captures an optical image of a subject formed by an optical adaptor.The imaging device 28 is an image sensor. The imaging device 28photo-electrically converts the optical image of the subject andgenerates an imaging signal. The lens 21 and the imaging device 28constitute a single-eye camera having a single viewpoint.

The CCU 9 drives the imaging device 28. An imaging signal output fromthe imaging device 28 is input into the CCU 9. The CCU 9 performspre-processing including amplification, noise elimination, and the likeon the imaging signal acquired by the imaging device 28. The CCU 9converts the imaging signal on which the pre-processing has beenexecuted into a video signal such as an NTSC signal.

The control device 10 includes a video-signal-processing circuit 12, aread-only memory (ROM) 13, a random-access memory (RAM) 14, a cardinterface 15, an external device interface 16, a control interface 17,and a central processing unit (CPU) 18.

The video-signal-processing circuit 12 performs predetermined videoprocessing on the video signal output from the CCU 9. For example, thevideo-signal-processing circuit 12 performs video processing related toimprovement of visibility. For example, the video processing is colorreproduction, gray scale correction, noise suppression, contourenhancement, and the like. For example, the video-signal-processingcircuit 12 combines the video signal output from the CCU 9 and a graphicimage signal generated by the CPU 18. The graphic image signal includesan image of the operation screen and the like. Thevideo-signal-processing circuit 12 outputs a combined video signal tothe display unit 5.

The ROM 13 is a nonvolatile recording medium on which a program for theCPU 18 to control the operation of the endoscope device 1 is recorded.The RAM 14 is a volatile recording medium that temporarily storesinformation used by the CPU 18 for controlling the endoscope device 1.The CPU 18 controls the operation of the endoscope device 1 based on theprogram recorded on the ROM 13.

A memory card 42 is connected to the card interface 15. The memory card42 is a recording medium that is attachable to and detachable from theendoscope device 1. The card interface 15 inputs control-processinginformation, image information, and the like stored on the memory card42 into the control device 10. In addition, the card interface 15records the control-processing information, the image information, andthe like generated by the endoscope device 1 on the memory card 42.

An external device such as a USB device is connected to the externaldevice interface 16. For example, a personal computer (PC) 41 isconnected to the external device interface 16. The external deviceinterface 16 transmits information to the PC 41 and receives informationfrom the PC 41. By doing this, the PC 41 can display information. Inaddition, by inputting an instruction into the PC 41, a user can performan operation related to control of the endoscope device 1.

A turning tool 43 may be used in order to rotate rotor blades inside aturbine. The turning tool 43 is a driving device that generates adriving force to rotate the rotor blades. The rotor blades rotate inresponse to the driving force generated by the turning tool 43. Theturning tool 43 is connected to the external device interface 16. Theexternal device interface 16 outputs control information used forcontrolling the turning tool 43 to the turning tool 43. In addition, theexternal device interface 16 inputs state information indicating thestate of the turning tool 43 into the control device 10.

The control interface 17 performs communication with the operation unit4, the endoscope unit 8, and the CCU 9 for operation control. Thecontrol interface 17 notifies the CPU 18 of information input into theoperation unit 4 by the user. The control interface 17 outputs controlsignals used for controlling the light source device and the bendingdevice to the endoscope unit 8. The control interface 17 outputs acontrol signal used for controlling the imaging device 28 to the CCU 9.

A program executed by the CPU 18 may be recorded on a computer-readablerecording medium. The program recorded on this recording medium may beread and executed by a computer other than the endoscope device 1. Forexample, the program may be read and executed by the PC 41. The PC 41may control the endoscope device 1 by transmitting control informationused for controlling the endoscope device 1 to the endoscope device 1 inaccordance with the program. Alternatively, the PC 41 may acquire avideo signal from the endoscope device 1 and may process the acquiredvideo signal.

As described above, the endoscope device 1 includes the imaging device28 and the CPU 18. The imaging device 28 images a subject and generatesan imaging signal. The imaging signal includes an image of the subject.Accordingly, the imaging device 28 acquires the image of the subjectgenerated by imaging the subject. The image is a two-dimensional image(2D image). The image acquired by the imaging device 28 is input intothe CPU 18 via the video-signal-processing circuit 12.

The insertion unit 2 constitutes an imaging apparatus (camera). Theimaging device 28 may be disposed in the main body unit 3, and anoptical fiber may be disposed in the insertion unit 2. Light incident onthe lens 21 may reach the imaging device 28 via the optical fiber. Aborescope may be used as a camera.

Turbines are used for aircraft engines or power generators. There aregas turbines, steam turbines, or the like. Hereinafter, a structure of agas turbine will be described. Hereinafter, the gas turbine will becalled a turbine.

A turbine includes a compressor section, a combustion chamber, and aturbine section. Air is compressed in the compressor section. Thecompressed air is sent to the combustion chamber. Fuel continuouslyburns in the combustion chamber, and gas of high pressure and hightemperature is generated. The gas expands in the turbine section andgenerates energy. A compressor rotates by using the energy, and the restof the energy is extracted. In the compressor section and the turbinesection, a rotor blade fixed to a rotation axis of an engine and astator blade fixed to a easing are alternately disposed.

The turbine includes a component disposed in a space inside the turbine.The component is a moving object capable of moving inside the turbine oris a stationary object that stands still inside the turbine. The movingobject is a rotor blade. The stationary object is a stator blade or ashroud.

FIG. 3 schematically shows the disposition of rotor blades and statorblades in a compressor section of a turbine TB10. FIG. 3 shows part of across-section of the turbine TB10 passing through a rotation axis RA10of an engine. The turbine TB10 includes a rotor blade RT10, a statorblade ST10, a rotor blade RT11, a stator blade ST11, a rotor blade RT12,a stator blade ST12, a rotor blade RT13, and a stator blade ST13 in thecompressor section. These rotor blades rotate in a direction DR12 aroundthe rotation axis RA10.

Air introduced into the turbine TB10 flows in a direction DR11. Therotor blade RT10 is disposed in a low-pressure section that introducesair. The rotor blade RT13 is disposed in a high-pressure section thatexpels air.

An access port AP10 is formed that enables an internal inspection of theturbine TB10 without disassembling the turbine TB10. The turbine TB10includes two or more access ports, and one of the two or mom accessports is shown as the access port AP10 in FIG. 3 . The access port AP10is a hole formed in the turbine TB10.

The insertion unit 2 constitutes an endoscope. The insertion unit 2 isinserted into the turbine TB10 through the access port AP10. When theinsertion unit 2 is inserted into the turbine TB10, the insertion unit 2moves in a direction DR10. When the insertion unit 2 is pulled out ofthe turbine TB10, the insertion unit 2 moves in an opposite direction tothe direction DR10. The direction DR10 is different from the directionDR12. Illumination light LT10 is emitted from the distal end 20 of theinsertion unit 2.

FIG. 4 schematically shows the disposition of two or more rotor bladesRT10 seen from a parallel direction to a rotation axis RA10. In FIG. 4 ,eight rotor blades RT10 are disposed in a disk DS10. The rotation axisRA10 passes through the center of the disk DS10. The disk DS10 rotatesin a direction DR12. Therefore, the eight rotor blades RT10 rotate inthe direction DR12.

Several tens of rotor blades or more than 100 rotor blades are actuallydisposed in one disk. The number of rotor blades in one disk depends onthe type of engine and also depends on the position of the disk in aregion ranging from a low-pressure section to a high-pressure section.

A user manually rotates a disk, or the turning tool 43 rotates the disk.The insertion unit 2 is inserted into the turbine TB10 through theaccess port AP10, and the distal end 20 is fixed. When the disk isrotating, the user performs an inspection of two or more rotor bladesand determines whether there is an abnormality in each rotor blade. Thisinspection is one of major inspection items in an inspection of aturbine.

FIG. 5 shows an example of an image acquired by the endoscope device 1.An image IMG10 shown in FIG. 5 is an image of a compressor section on ahigh-pressure side. A rotor blade RT14, a stator blade ST14, and ashroud SH10 are seen in the image IMG10.

When a subject is seen from the distal end 20 of the insertion unit 2,the stator blade ST14 is disposed in front of the rotor blade RT14 andthe shroud SH10 is disposed at the rear of the rotor blade RT14. Thedistance between the distal end 20 and the stator blade ST14 is lessthan that between the distal end 20 and the rotor blade RT14. Thedistance between the distal end 20 and the shroud SH10 in a region inwhich the rotor blade RT14 conceals the shroud SH10 is greater than thatbetween the distal end 20 and the rotor blade RT14. The stator bladeST14 conceals part of the rotor blade RT14.

The imaging device 28 generates two or more images. Each of the two ormore images is temporally associated with the other images included inthe two or more images. For example, each of the two or more images is astill image. A video may be used instead of the still image. Two or moreframes included in the video are associated with each other bytimestamps (timecodes).

The RAM 14 stores the two or more images generated by the imaging device28. When the disk is rotating, a relative position of a rotor blade tothe distal end 20 (viewpoint) of the insertion unit 2 is differentbetween the two or more images. Alternatively, a relative position andposture of a rotor blade to the distal end 20 are different between thetwo or more images. In other words, the position of the rotor blade isdifferent between timings at which the imaging device 28 generatesimages. Therefore, the position (two-dimensional coordinates) of therotor blade in an image generated by the imaging device 28 changes. Whenthe disk stands still, the position (two-dimensional coordinates) of therotor blade in an image generated by the imaging device 28 does notchange.

In addition, the RAM 14 stores necessary parameters for 3D restorationprocessing. The parameters include an internal parameter of a camera, adistortion correction parameter of the camera, a setting value, scaleinformation, and the like. The setting value is used for various kindsof processing of generating three-dimensional data (3D data) indicatinga three-dimensional shape (3D shape) of a subject. The scale informationis used for converting the scale of the 3D data into an actual scale ofthe subject.

The memory card 42 may store the two or more images and theabove-described parameters. The endoscope device 1 may read the two ormore images and the parameters from the memory card 42 and may store thetwo or more images and the parameters on the RAM 14.

The endoscope device 1 may perform wireless or wired communication withan external device via the external device interface 16. The externaldevice is the PC 41, a cloud server, or the like. The endoscope device 1may transmit the two or more images generated by the imaging device 28to the external device. The external device may store the two or moreimages and the above-described parameters. The endoscope device 1 mayreceive the two or more images and the parameters and store the two ormore images and the parameters on the RAM 14.

FIG. 6 shows a functional configuration of the CPU 18. The CPU 18functions as a control unit 180, an image acquisition unit 181, a regiondetection unit 182, a region determination unit 183, a 3D restorationunit 184, and a display control unit 185. At least one of the blocksshown in FIG. 6 may be constituted by a different circuit from the CPU18.

The control unit 180 controls processing executed by each unit shown inFIG. 6 .

The image acquisition unit 181 acquires the two or more images and theabove-described parameters from the RAM 14. The image acquisition unit181 may acquire the two or more images and the above-describedparameters from the memory card 42 or the external device through theexternal device interface 16.

The region detection unit 182 detects two or more feature regions ineach of the two or more images. In addition, the region detection unit182 detects the same feature regions (correspondence regions) in the twoor more images. A correspondence region is a region of a componentincluded in a turbine.

For example, in a case in which a first feature region in a first imageand a second feature region in a second image are the same, the regiondetection unit 182 associates the first feature region and the secondfeature region with each other as a correspondence region. In a case inwhich the second feature region is the same as a third feature region ina third image, the region detection unit 182 associates the secondfeature region and the third feature region with each other as acorrespondence region. In this case, the region detection unit 182detects the same correspondence region between three images.

The region determination unit 183 determines whether a feature region ineach of the two or more images is a moving region or a stationaryregion. By doing this, the region determination unit 183 classifies thefeature region into the moving region or the stationary region. Theregion determination unit 183 may determine whether each of featureregions included in only part of each image is the moving region or thestationary region. The region determination unit 183 may determinewhether each of feature regions included in the entire image is themoving region or the stationary region. Each image may include themoving region and the stationary region. Alternatively, each image mayinclude only the moving region or only the stationary region.

The 3D restoration unit 184 executes the 3D restoration processing byusing correspondence regions determined to be moving regions amongfeature regions of the two or more images and generates 3D data. At thistime, the 3D restoration unit 184 does not use correspondence regionsdetermined to be stationary regions among the feature regions of the twoor more images.

The 3D data include three-dimensional coordinates (3D coordinates) oftwo or more regions of a subject and also include camera coordinate andposture information. The camera coordinate indicates 3D coordinates of acamera that has acquired each of two or more images and is associatedwith each of the two or more images. The camera coordinate indicates 3Dcoordinates of a viewpoint when each image is acquired. For example, thecamera coordinate indicates 3D coordinates of an observation opticalsystem included in the camera. The posture information indicates aposture of the camera that has acquired each of the two or more imagesand is associated with each of the two or more images. For example, theposture information indicates a posture of the observation opticalsystem included in the camera.

The display control unit 185 controls processing executed by thevideo-signal-processing circuit 12. The CCU 9 outputs a video signal.The video signal includes color data of each pixel of an image acquiredby the imaging device 28. The display control unit 185 causes thevideo-signal-processing circuit 12 to output the video signal outputfrom the CCU 9 to the display unit 5. The video-signal-processingcircuit 12 outputs the video signal to the display unit 5. The displayunit 5 displays an image based on the video signal output from thevideo-signal-processing circuit 12. By doing this, the display controlunit 185 displays the image acquired by the imaging device 28 on thedisplay unit 5.

The display control unit 185 displays various kinds of information onthe display unit 5. In other words, the display control unit 185displays various kinds of information on an image.

For example, the display control unit 185 generates a graphic imagesignal of the various kinds of information. The display control unit 185outputs the generated graphic image signal to thevideo-signal-processing circuit 12. The video-signal-processing circuit12 combines the video signal output from the CCU 9 and the graphic imagesignal output from the CPU 18. Due to this, the various kinds ofinformation are superimposed on an image. The video-signal-processingcircuit 12 outputs the combined video signal to the display unit 5. Thedisplay unit 5 displays an image on which the various kinds ofinformation are superimposed.

In addition, the display control unit 185 generates a graphic imagesignal of 3D data. The display control unit 185 outputs the graphicimage signal to the video-signal-processing circuit 12. Similarprocessing to that described above is executed, and the display unit 5displays an image of the 3D data. By doing this, the display controlunit 185 displays the image of the 3D data on the display unit 5.

Each unit shown in FIG. 6 may be constituted by at least one of aprocessor and a logic circuit. For example, the processor is at leastone of a central processing unit (CPU), a digital signal processor(DSP), and a graphics-processing unit (GPU). For example, the logiccircuit is at least one of an application-specific integrated circuit(ASIC) and a field-programmable gate array (FPGA). Each unit shown inFIG. 6 may include one or a plurality of processors. Each unit shown inFIG. 6 may include one or a plurality of logic circuits.

A computer of the endoscope device 1 may read a program and may executethe read program. The program includes commands defining the operationsof each unit shown in FIG. 6 . In other words, the functions of eachunit shown in FIG. 6 may be realized by software.

The program described above, for example, may be provided by using a“computer-readable storage medium” such as a flash memory. The programmay be transmitted from the computer storing the program to theendoscope device 1 through a transmission medium or transmission wavesin a transmission medium. The “transmission medium” transmitting theprogram is a medium having a function of transmitting information. Themedium having the function of transmitting information includes anetwork (communication network) such as the Internet and a communicationcircuit line (communication line) such as a telephone line. The programdescribed above may realize some of the functions described above. Inaddition, the program described above may be a differential file(differential program). The functions described above may be realized bya combination of a program that has already been recorded in a computerand a differential program.

Hereinafter, distinctive processing of the first embodiment will bedescribed. In the following descriptions, it is assumed that 3D data aregenerated by using two or more images acquired by endoscope equipment.Inspection equipment that acquires two or more images is not limited tothe endoscope equipment. As long as an image of a component inside aturbine is acquired by using equipment including a camera, any equipmentmay be used.

The control device 10 functions as a 3D data generation device. A 3Ddata generation device according to each aspect of the present inventionmay be a computer system such as a PC other than endoscope equipment.The 3D data generation device may be any one of a desktop PC, a laptopPC, and a tablet terminal. The 3D data generation device may be acomputer system that operates on a cloud.

Processing executed by the endoscope device 1 will be described by usingFIG. 7 . FIG. 7 shows a procedure of 3D data generation processingexecuted by the endoscope device 1.

When the processing shown in FIG. 7 is started, the control unit 180sets a number n used for managing an image acquired from the RAM 14 to 0(Step S100).

After Step S100, the control unit 180 increases the number n by 1 inorder to acquire an image (Step S101). When Step S101 is executed forthe first time, the number n is set to 1.

After Step S101, the image acquisition unit 181 acquires an image IMGnindicated by the number n from the RAM 14 (Step S102). When Step S102 isexecuted for the first time, the image acquisition unit 181 acquires animage IMG1.

After Step S102, the region detection unit 182 analyzes the image IMGnand detects two or more feature regions seen in the image IMGn (StepS103).

The feature region indicates a corner, an edge, or the like in which animage luminance gradient is large among regions seen in an image. Thefeature region may be constituted by one pixel, which is a minimum unitof an image. Alternatively, the feature region may be constituted by twoor more pixels. The region detection unit 182 detects a feature regionby using scale-invariant feature transform (SIFT), features fromaccelerated segment test (FAST), or the like.

FIG. 8 shows an example of a feature region. An image IMGn is shown inFIG. 8 . The region detection unit 182 detects feature regions P1 n, P2n, P3 n, P4 n, P5 n, and P6 n in the image IMGn.

After Step S103, the control unit 180 determines whether the number n is1 (Step S104).

In the first embodiment, the region determination unit 183 calculates amoving amount of a region of a subject between two images in Step S106described later. Therefore, two images having at least differentphotography time points need to be acquired from the RAM 14. When thecontrol unit 180 determines that the number n is 1 in Step S104, StepS101 is executed. When the control unit 180 determines that the number nis not 1 in Step S104, Step S105 described later is executed.

When the number n is two or more, two or more feature regions havealready been detected in each of an image IMG(n−1) and the image IMGn. Atiming at which the imaging device 28 generates the image IMGn isdifferent from that at which the imaging device 28 generates the imageIMG(n−1). The region detection unit 182 detects the same feature regions(correspondence regions) in the image IMG(n−1) and the image IMGn (StepS105).

The region detection unit 182 executes the following processing in StepS105. The region detection unit 182 calculates a correlation degree of afeature region between the image IMG(n−1) and the image IMGn. When theregion detection unit 182 has found a feature region having a highcorrelation degree between the two images, the region detection unit 182holds information (correspondence information) of the feature region(correspondence region) on the RAM 14. By doing this, the regiondetection unit 182 associates feature regions of the two images witheach other. On the other hand, in a case in which the region detectionunit 182 has found no feature region having a high correlation degreebetween the two images, the region detection unit 182 discardsinformation of the correspondence region between the image IMG(n−1) andthe image IMGn.

FIG. 9 shows an example of the same feature regions between two images.Two or more feature regions in an image IMG(n−1) and two or more featureregions in an image IMGn are shown in FIG. 9 . After the image IMGnshown in FIG. 8 is acquired in Step S102, the number n increases by 1 inStep S101. Thereafter, the image IMGn shown in FIG. 9 is acquired inStep S102. The image IMG(n−1) shown in FIG. 9 is the same as the imageIMGn shown in FIG. 8 .

Correspondence information M1, M2, M3, M4, M5, and M6 indicate the samefeature regions between the image IMG(n−1) and the image IMGn. Thecorrespondence information M1 indicates that a feature region P1(n−1) inthe image IMG(n−1) is the same as a feature region P1 n in the imageIMGn. The correspondence information M2 indicates that a feature regionP2(n−1) in the image IMG(n−1) is the same as a feature region P2 n inthe image IMGn. The correspondence information M3 indicates that afeature region P3(n−1) in the image IMG(n−1) is the same as a featureregion P3 n in the image IMGn. The correspondence information M4indicates that a feature region P4(n−1) in the image IMG(n−1) is thesame as a feature region P4 n in the image IMGn. The correspondenceinformation M5 indicates that a feature region P5(n−1) in the imageIMG(n−1) is the same as a feature region P5 n in the image IMGn. Thecorrespondence information M6 indicates that a feature region P6(n−1) inthe image IMG(n−1) is the same as a feature region P6 n in the imageIMGn.

A feature region in one image is not always the same as that in anotherimage. For example, there is a possibility that a feature regiondetected in the edge part of the image IMG(n−1) is outside the visualfield of the camera when the image IMGn is acquired. In addition, it maybe difficult to associate feature regions with each other between twoimages due to an influence of blurring or the like. Therefore, thenumber of feature regions in each image is greater than or equal to thenumber of the same feature regions as those in other images.

After Step S105, the region determination unit 183 calculates a movingamount of a region of a subject between the image IMG(n−1) and the imageIMGn (Step S106).

The region determination unit 183 executes the following processing inStep S106. The region determination unit 183 divides the region of theimage TMGn into two or more grid-like small regions as shown in FIG. 10. In the example shown in FIG. 10 , the region of the image IMGn isdivided into seven in the horizontal direction and is divided into sixin the vertical direction. The region of the image IMGn is divided into42 small regions. The shape of the small regions is not a grid. Theshape of the small regions may be a circle or the like. The regiondetermination unit 183 divides the region of the image IMG(n−1) into twoor more grid-like small regions by using a similar method as thatdescribed above.

The region determination unit 183 refers to correspondence informationof a feature region corresponding to a specific small region. Thecorrespondence information indicates the same feature regions betweentwo images. The region determination unit 183 calculates a typicalmoving amount between a small region of the image IMG(n−1) and a smallregion of the image IMGn. The small region of the image IMG(n−1) and thesmall region of the image IMGn are associated with each other in thecorrespondence information.

The region determination unit 183 can use a statistic as a typicalmoving amount. The statistic is an average, a median, or the like.

The region determination unit 183 may determine the reliability of themoving amount based on the deviation of moving amounts of two or morefeature regions included in a small region or the deviation of movingdirections of the two or more feature regions. For example, when movingdirections are not uniform, there is a possibility that the same featureregions are not correctly associated with each other between two images.In such a case, the region determination unit 183 may determine that thereliability of the moving amount is low. When the moving amount or themoving direction regarding a small region is different from thatregarding the other small regions, the region determination unit 183 maydetermine that the moving amount is abnormal and may exclude the movingamount.

The region determination unit 183 executes the above-describedprocessing by using all the small regions of the image IMGn andcalculates a typical moving amount of each of the small regions.

After Step S106, the region determination unit 183 determines whether asmall region of the image IMGn is a moving region or a stationary regionbased on the moving amount (Step S107).

The region determination unit 183 executes the following processing inStep S107. The region determination unit 183 compares a typical movingamount of each small region with a threshold value. The threshold valueis set in advance. Alternatively, the threshold value is calculated inaccordance with the situation of an inspection. When the moving amountis greater than the threshold value, the region determination unit 183determines that the small region is the moving region. When the movingamount is less than or equal to the threshold value, the regiondetermination unit 183 determines that the small region is thestationary region.

After Step S107, the 3D restoration unit 184 extracts a feature regionincluded in the small region determined to be the moving region in StepS107 from the region of the image IMGn. By doing this, the 3Drestoration unit 184 extracts a feature region corresponding to themoving region (Step S108).

The 3D restoration unit 184 does not extract a feature region includedin the small region determined to be the stationary region in Step S107from the region of the image IMGn. In other words, the 3D restorationunit 184 does not extract a feature region corresponding to thestationary region. Therefore, the feature region corresponding to thestationary region is not used in the 3D restoration processing.

Details of Step S107 and Step S108 will be described by using FIG. 11 .Two or more feature regions in an image IMG(n−1) and two or more featureregions in an image IMGn are shown in FIG. 11 . Each of the featureregions is the same as that shown in FIG. 9 .

The region determination unit 183 calculates a moving amount of eachsmall region of the image IMG(n−1) by using an image IMG(n−2) and theimage IMG(n−1). The region determination unit 183 classifies two or morefeature regions of the image IMG(n−1) into a moving region and astationary region MS(n−1) based on the moving amount. The moving regionincludes feature regions P2(n−1), P3(n−1), P4(n−1), and P5(n−1).

The region determination unit 183 calculates a moving amount of eachsmall region of the image IMGn by using the image IMG(n−1) and the imageIMGn. The region determination unit 183 classifies two or more featureregions of the image IMGn into a moving region and a stationary regionMSn based on the moving amount. The moving region includes featureregions P2 n, P3 n, P4 n, and P5 n.

The 3D restoration unit 184 extracts a feature region corresponding tothe moving region in Step S108. The 3D restoration unit 184 does notextract a feature region corresponding to the stationary region MS(n−1)or the stationary region MSn in Step S108.

Similar processing to that described above is executed each time a newimage is acquired in Step S102.

After Step S108, the 3D restoration unit 184 executes the 3D restorationprocessing by using the feature region corresponding to the movingregion (Step S109). The 3D restoration unit 184 reads a necessaryparameter for the 3D restoration processing from the RAM 14 and uses theparameter in the 3D restoration processing.

The 3D restoration unit 184 executes the following processing in StepS109. FIG. 12 schematically shows a situation of image acquisition in acase in which two images are acquired.

As shown in FIG. 12 , first, an image I₁ is acquired in an imaging statec₁ of the camera. Next, an image I₂ is acquired in an imaging state c₂of the camera. At least one of an imaging position and an imagingposture is different between the imaging state c₁ and the imaging statec₂. In FIG. 12 , both the imaging position and the imaging posture aredifferent between the imaging state c₁ and the imaging state c₂.

In each embodiment of the present invention, it is assumed that theimage I₁ and the image I₂ are acquired by the same endoscope. Inaddition, in each embodiment of the present invention, it is assumedthat parameters of an objective optical system of the endoscope do notchange. The parameters of the objective optical system are a focaldistance, a distortion aberration, a pixel size of an image sensor, andthe like. Hereinafter, for the convenience of description, theparameters of the objective optical system will be abbreviated asinternal parameters. When such conditions are assumed, the internalparameters specifying characteristics of the optical system of theendoscope can be used in common regardless of the position and theposture of the camera (observation optical system). In each embodimentof the present invention, it is assumed that the internal parameters areacquired at the time of factory shipment. In addition, in eachembodiment of the present invention, it is assumed that the internalparameters are known at the time of acquiring an image.

For example, the image I₁ and the image I₂ are still images. The imageI₁ and the image L may be specific frames extracted from a video. Ineach embodiment of the present invention, it is assumed that the image hand the image I₂ are acquired by one endoscope. However, the presentinvention is not limited to this. For example, the present invention maybe also applied to a case in which 3D data are generated by using aplurality of videos acquired by a plurality of endoscopes. In this case,the image I₁ and the image I₂ have only to be acquired by usingdifferent endoscope devices, and each internal parameter has only to bestored for each endoscope. Even if the internal parameters are unknown,it is possible to perform calculation by using the internal parametersas variables. Therefore, the subsequent procedure does not greatlychange in accordance with whether the internal parameters are known.

FIG. 12 shows an example in which a feature region at coordinates P₁₁ isextracted and a feature region at coordinates P₁₂ are extracted. Thefeature region at the coordinates P₁₁ and the feature region at thecoordinates P₁₂ correspond to a region at coordinates P₁ on a subject.These feature regions are detected in Step S103 and are associated witheach other in Step S105.

Although only one feature region of each image is shown in FIG. 12 , twoor more feature regions are actually detected in each image. The numberof feature regions to be detected may be different between images. Eachfeature region detected from each image is converted into data called afeature. The feature indicates data representing characteristics of afeature region.

The 3D restoration processing in Step S109 will be described by usingFIG. 13 . FIG. 13 shows a procedure of the 3D restoration processing.

The 3D restoration unit 184 reads coordinates of feature regionsassociated with each other between two images from the RAM 14. Thecoordinates are a pair of coordinates of feature regions in each image.The 3D restoration unit 184 executes position-and-posture calculationprocessing based on the read coordinates (Step S109 a). In theposition-and-posture calculation processing, the 3D restoration unit 184calculates a relative position and a relative posture between theimaging state c₁ of the camera that acquires the image I₁ and theimaging state c₂ of the camera that acquires the image I₂. Morespecifically, the 3D restoration unit 184 calculates a matrix E bysolving the following Equation (1) using an epipolar restriction.

$\begin{matrix}{{{p_{1}}^{T}{Ep}_{2}} = {{0E} = {{{\lbrack t\rbrack_{X}R}\because\lbrack t\rbrack_{X}} = \begin{pmatrix}0 & {- t_{z}} & t_{y} \\t_{z} & 0 & {- t_{x}} \\{- t_{y}} & t_{x} & 0\end{pmatrix}}}} & (1)\end{matrix}$

The matrix E is called a basic matrix. The basic matrix E is a matrixstoring a relative position and a relative posture between the imagingstate c₁ of the camera that acquires the image I₁ and the imaging statec₂ of the camera that acquires the image I₂. In Equation (1), a matrixp₁ is a matrix including coordinates of a feature region detected fromthe image I₁. A matrix p₂ is a matrix including coordinates of a featureregion detected from the image I₂. The basic matrix E includesinformation related to a relative position and a relative posture of thecamera and thus corresponds to external parameters of the camera. The 3Drestoration unit 184 can solve the basic matrix E by using a knownalgorithm.

As shown in FIG. 12 , Expression (2) and Expression (3) are satisfied ina case in which the amount of position (relative position) change of thecamera is t and the amount of posture (relative posture) change of thecamera is R.

$\begin{matrix}{t = \left( {t_{x},t_{y},t_{z}} \right)} & (2)\end{matrix}$ $\begin{matrix}{R = {{{R_{x}(\alpha)}{R_{y}(\beta)}{R_{z}(\gamma)}} = {\begin{pmatrix}1 & 0 & 0 \\0 & {\cos\alpha} & {{- \sin}\alpha} \\0 & {\sin\alpha} & {\cos\alpha}\end{pmatrix}\begin{pmatrix}{\cos\beta} & 0 & {\sin\beta} \\0 & 1 & 0 \\{{- \sin}\beta} & 0 & {\cos\beta}\end{pmatrix}\begin{pmatrix}{\cos\gamma} & {{- \sin}\gamma} & 0 \\{\sin\gamma} & {\cos\gamma} & 0 \\0 & 0 & 1\end{pmatrix}}}} & (3)\end{matrix}$

In Expression (2), a moving amount in an x-axis direction is expressedas t_(x), a moving amount in a y-axis direction is expressed as t_(y),and a moving amount in a z-axis direction is expressed as t_(z). InExpression (3), a rotation amount α around the x-axis is expressed asR_(x)(α), a rotation amount β around the y axis is expressed asR_(y)(β), and a rotation amount γ around the z axis is expressed asR_(z)(γ). After the basic matrix E is calculated, optimizationprocessing called bundle adjustment may be executed in order to improverestoration accuracy of 3D coordinates.

The 3D restoration unit 184 calculates 3D coordinates (cameracoordinate) in a coordinate system of 3D data by using the calculatedamount of positional change of the camera. For example, the 3Drestoration unit 184 defines 3D coordinates of the camera that acquiresthe image I₁. The 3D restoration unit 184 calculates 3D coordinates ofthe camera that acquires the image I₂ based on both the 3D coordinatesof the camera that acquires the image I₁ and the amount of positionalchange of the camera that acquires the image I₂.

The 3D restoration unit 184 calculates posture information in acoordinate system of 3D data by using the calculated amount of posturalchange of the camera. For example, the 3D restoration unit 184 definesposture information of the camera that acquires the image I₁. The 3Drestoration unit 184 generates posture information of the camera thatacquires the image I₂ based on both the posture information of thecamera that acquires the image I₁ and the amount of postural change ofthe camera that acquires the image I₂.

The 3D restoration unit 184 generates data (3D shape data) of athree-dimensional shape (3D shape) by executing the position-and-posturecalculation processing (Step S109 a). The 3D shape data include 3Dcoordinates (camera coordinate) at the position of the camera andinclude posture information indicating the posture of the camera. Inaddition, in a case in which a method such as structure from motion,visual-SLAM, or the like is applied to the position-and-posturecalculation processing (Step S109 a), the 3D restoration unit 184further calculates 3D coordinates of each feature region in Step S109 a.The 3D shape data generated in Step S109 a do not include 3D coordinatesof a region on the subject other than the feature region. Therefore, the3D shape data indicate a sparse 3D shape of the subject.

The 3D shape data include the 3D coordinates of each feature region, theabove-described camera coordinate, and the above-described postureinformation. The 3D coordinates of each feature region are defined inthe coordinate system of the 3D data. The 3D coordinates of each featureregion are associated with two-dimensional coordinates (2D coordinates)of each feature region. The 2D coordinates of each feature region aredefined in a coordinate system of an image including each featureregion. The 2D coordinates and the 3D coordinates of each feature regionare associated with an image including each feature region.

After Step S109 a, the 3D restoration unit 184 executesthree-dimensional shape restoration processing based on the relativeposition and the relative posture of the camera (the amount t ofpositional change and the amount R of postural change) calculated inStep S109 a (Step S109 b). The 3D restoration unit 184 generates 3D dataof the subject in the three-dimensional shape restoration processing. Asa technique for restoring a three-dimensional shape of the subject,there are patch-based multi-view stereo (PMVS), matching-processing thatuses parallelization stereo, and the like. However, a means therefor isnot particularly limited.

The 3D restoration unit 184 calculates 3D coordinates of regions on thesubject other than feature regions in Step S109 b. The 3D coordinates ofeach region other than the feature regions are defined in the coordinatesystem of the 3D data. The 3D coordinates of each region are associatedwith the 2D coordinates of each region. The 2D coordinates of eachregion are defined in a coordinate system of a 2D image including eachregion. The 2D coordinates and the 3D coordinates of each region areassociated with a 2D image including each region. The 3D restorationunit 184 updates the 3D shape data. The updated 3D shape data includethe 3D coordinates of each feature region, the 3D coordinates of eachregion other than the feature regions, the camera coordinate, and theposture information. The 3D shape data updated in Step S109 b include 3Dcoordinates of a region on the subject other than the feature regions inaddition to the 3D coordinates of the feature regions. Therefore, the 3Dshape data indicate a dense 3D shape of the subject.

After Step S109 b, the 3D restoration unit 184 executes scale conversionprocessing based on both the 3D shape data processed in thethree-dimensional shape restoration processing (Step S109 b) and thescale information read from the RAM 14 (Step S109 c). The 3D restorationunit 184 transforms the 3D shape data of the subject intothree-dimensional coordinate data (3D data) having a dimension of lengthin the scale conversion processing. When Step S109 c is executed, the 3Drestoration processing is completed.

In order to shorten a processing time, Step S109 b may be omitted. Inthis case, after Step S109 a is executed, Step S109 c is executedwithout executing Step S109 b.

Step S109 c may be omitted. In this case, after Step S109 b is executed,the 3D restoration processing is completed without executing Step S109c. In this case, the 3D data indicate a relative shape of the subjectnot having a dimension of length.

It is necessary that at least part of a region of one of the images andat least part of a region of at least one of the other images overlapeach other in order to generate 3D data in accordance with the principleshown in FIG. 12 . In other words, a region of a first image and aregion of a second image different from the first image include a commonregion. The other region in the first image excluding the common regionand the other region in the second image excluding the common region aredifferent from each other.

For example, the 3D restoration unit 184 generates 3D data by using afeature region of the image IMG(n−1) and a feature region of the imageIMGn in Step S109. The feature region of each image corresponds to themoving region extracted in Step S108. When Step S109 has been executedtwice or more, the 3D restoration unit 184 combines the 3D datagenerated in Step S109 executed last time and the 3D data generated inStep S109 executed this time.

The image I₁ and the image I₂ do not need to be two temporallyconsecutive frames in a video. There may be one or more frames betweenthe image I₁ and the image b in the video.

The processing executed by the endoscope device 1 will be described byusing FIG. 7 again.

After Step S109, the control unit 180 determines whether the number nhas reached a predetermined number. By doing this, the control unit 180determines whether all images are acquired (Step S110).

For example, the predetermined number is a number of the last frame of avideo or a number set on software in advance. The predetermined numbermay be a number of a frame that is processed when a user performs acompletion operation. In this case, the predetermined number varies inaccordance with a timing at which the user performs the completionoperation.

When the control unit 180 determines that the number n has not reachedthe predetermined number in Step S110, Step S101 is executed. In thiscase, a new image is acquired in Step S102 and the processing describedabove is repeated. When the control unit 180 determines that the numbern has reached the predetermined number in Step S110, the display controlunit 185 displays an image of the 3D data generated in Step S109 on thedisplay unit 5 (Step S111). When Step S111 is executed, the 3D datageneration processing is completed.

The endoscope device 1 may store the generated 3D data on the RAM 14 orthe memory card 42 instead of executing Step S111. Alternatively, theendoscope device 1 may transmit the generated 3D data to an externaldevice. The external device is the PC 41, a cloud server, or the like.

After Step S111 is executed, a user may designate a measurement positionin the 3D data displayed on the display unit 5. The endoscope device 1may execute measurement by using the 3D data.

In the above-described example, the determination regarding a movingregion and a stationary region is executed in Step S107 after two ormore feature regions are detected in Step S103. After two or moretemporary feature regions are detected in Step S103 and thedetermination regarding a moving region and a stationary region isexecuted in Step S107, a formal feature region may be calculated basedon information of the moving region.

An example of a user interface regarding the 3D data generationprocessing will be described by using FIG. 14 and FIG. 15 . FIG. 14 andFIG. 15 show an example of information displayed on the display unit 5.

The display control unit 185 displays a dialog box DB10 shown in FIG. 14on the screen of the display unit 5. The dialog box DB10 includes aregion RG10, a region RG11, a button BT10, a button BT11, and a seek-barSB10.

An image of a subject is displayed in the region RG10. In the examplesshown in FIG. 14 and FIG. 15 , a video of the subject is displayed inthe region RG10. An image of 3D data is displayed in the region RG11.

A user operates the button BT10 and the button BT11 by operating theoperation unit 4. In a case in which the display unit 5 is constitutedas a touch panel, the user operates the button BT10 and the button BT11by touching the screen of the display unit 5.

The user presses the button BT10 in order to read a video from the RAM14. After the button BT10 is pressed, a frame of the video is displayedin the region RG10. FIG. 14 shows the dialog box DB10 after the buttonBT10 is pressed.

The user may perform a predetermined operation on the region RG10 byoperating the operation unit 4 or touching the screen of the displayunit 5. When the predetermined operation is performed, an instruction toreproduce or pause the video may be input into the endoscope device 1.The dialog box DB10 may include a button used for inputting theinstruction to reproduce or pause the video.

The seek-bar SB10 indicates the position of the frame displayed in theregion RG10. The user can change the position of the frame in theseek-bar SB10 by operating the operation unit 4 or touching the screenof the display unit 5. In addition, the user can designate a frame FR10for which the 3D restoration processing is started and a frame FR11 forwhich the 3D restoration processing is completed by operating theoperation unit 4 or touching the screen of the display unit 5.

In the above-described example, the user designates a start frame forwhich the 3D restoration processing is started and a completion framefor which the 3D restoration processing is completed. The control unit180 may automatically designate the start frame and the completionframe. For example, the control unit 180 may detect a section of thevideo in which a subject is moving. Alternatively, the control unit 180may detect a section of the video in which an abnormality such as damageis seen. The section includes two or more frames of the video. Thecontrol unit 180 may designate the initial frame of the section as thestart frame and may designate the last frame of the section as thecompletion frame.

Only one of the start frame and the completion frame may be designatedby the user. Alternatively, only one of the start frame and thecompletion frame may be automatically designated. A method of setting asection including a frame to be used in the 3D restoration processing isnot limited to the above-described examples.

The user presses the button BT11 in order to start the 3D restorationprocessing. After the button BT11 is pressed, the 3D data generationprocessing shown in FIG. 7 is started. FIG. 15 shows the dialog box DB10after the button BT11 is pressed.

After the button BT11 is pressed, the button BT11 is changed to a buttonBT12 used to suspend the 3D restoration processing. When the 3D datageneration processing is being executed, the user can input aninstruction to suspend the 3D data generation processing into theendoscope device 1 by pressing the button BT12 at any timing. When theuser presses the button BT12, the control unit 180 sets the presentnumber n as the predetermined number used in Step S110. After processingrelated to the present number n is completed, Step S111 is executed.

The seek-bar SB10 shows a frame FR12 corresponding to the present numbern. The user can check the progress of processing from the frame FR 10for which the 3D restoration processing is started to the frame FR12 forwhich the 3D restoration processing is suspended.

The display control unit 185 displays an image 3D10 of 3D data in theregion RG11. The image 3D10 indicates 3D data generated by using theframes FR10 to FR12. In the example shown in FIG. 15 , the 3D dataindicate 3D shapes of seven restored rotor blades. In the example shownin FIG. 15 , a number is allocated to each rotor blade, and the numberis displayed. In the example shown in FIG. 15 , eight rotor blades aredisposed, and 3D data of the one remaining rotor blade have not beengenerated yet.

As a method of allocating numbers to rotor blades, any method may beused. Hereinafter, an example of a method of allocating numbers to rotorblades will be described.

For example, the 3D restoration unit 184 may recognize rotor blades inan image displayed in the region RG10 by using an image recognitiontechnique such as machine learning and may allocate a number to eachrotor blade in accordance with the number of recognized rotor blades. Ina case in which the turning tool 43 rotates the rotor blades, the 3Drestoration unit 184 may acquire information indicating rotation anglesof the rotor blades from the turning tool 43. The 3D restoration unit184 may identify rotor blades seen in the image displayed in the regionRG10 based on the rotation angles of the rotor blades and the number ofrotor blades fixed on the circumference of the disk. The 3D restorationunit 184 may allocate numbers to the identified rotor blades. The numberof rotor blades fixed on the circumference of the disk is known.

The 3D restoration unit 184 may apply a shape recognition technique tothe generated 3D data and may recognize an object having a similar shapeto that of a rotor blade. The 3D restoration unit 184 may allocate anumber to the object.

In a case in which the 3D data generation device is a device other thanthe endoscope device 1, the control unit 180 may transmit two or moreimages generated by the imaging device 28 to the 3D data generationdevice via the external device interface 16. The 3D data generationdevice may receive the two or more images and may execute similarprocessing to that shown in FIG. 7 .

A 3D data generation method according to each aspect of the presentinvention generates 3D data indicating a 3D shape inside a turbine. The3D data generation method includes an image acquisition step, a regiondetection step, a region determination step, and a data generation step.

The CPU 18 acquires two or more images of a component inside the turbinein the image acquisition step (Step S102). The component includes afirst object capable of moving inside the turbine and a second objectthat is stationary inside the turbine. The two or more images aregenerated by an imaging apparatus including the tubular insertion unit 2that acquires an optical image inside the turbine. The insertion unit 2is inserted into the turbine through a hole (access port AP10) formed inthe turbine. A moving direction (direction DR10) of the insertion unit 2when the insertion unit 2 is inserted into the turbine is different froma moving direction (direction DR12) of the first object. A relativeposition of the first object to the insertion unit 2 is differentbetween timings at which the imaging apparatus generates images whilethe first object moves.

The CPU 18 detects two or more correspondence regions that are the sameregions of the component in at least two images included in the two ormore images in the region detection step (Step S105). The CPU 18determines whether at least part of a region of each of the two or moreimages is a change region (moving region) or a non-change region(stationary region) in the region determination step (Step S107). Thechange region is a region of the component of which coordinates in animage generated by the imaging apparatus change. The non-change regionis a region of the component of which coordinates in an image generatedby the imaging apparatus do not change. The CPU 18 generates the 3D databy using a correspondence region determined to be the change regionamong the two or more correspondence regions without using acorrespondence region determined to be the non-change region among thetwo or more correspondence regions in the data generation step (StepS109).

A 3D data generation system according to each aspect of the presentinvention includes an imaging apparatus and a 3D data generation device(control device 10). The imaging apparatus includes the tubularinsertion unit 2, which acquires an optical image inside the turbine,and generates two or more images of a component inside the turbine. The3D data generation device includes the CPU 18. The CPU 18 executes theimage acquisition step, the region detection step, the regiondetermination step, and the data generation step described above.

Each aspect of the present invention may include the following modifiedexample. After the CPU 18 detects the two or more correspondence regionsin the region detection step (Step S105), the CPU 18 determines whetherat least part of a region in each of the two or more images is thechange region or the non-change region in the region determination step(Step S107).

Each aspect of the present invention may include the following modifiedexample. The first object includes a rotor blade. The second objectincludes a stator blade or a shroud.

Each aspect of the present invention may include the following modifiedexample. Part of the first object is concealed by the second object inthe two or more images. In the example shown in FIG. 5 , part of therotor blade RT14 is concealed by the stator blade ST14.

Each aspect of the present invention may include the following modifiedexample. The second object includes an object that conceals part of thefirst object in the two or more images. The second object includes anobject, part of which is concealed by the first object in the two ormore images. In the example shown in FIG. 5 , the stator blade ST14conceals part of the rotor blade RT14, and part of the shroud SH10 isconcealed by the rotor blade RT14.

Each aspect of the present invention may include the following modifiedexample. The imaging apparatus generates the two or more images at twoor more different timings. The CPU 18 determines whether at least partof a region in each of the two or more images is the change region orthe non-change region based on a moving amount of the correspondenceregion between at least two images included in the two or more images inthe region determination step (Step S107).

Each aspect of the present invention may include the following modifiedexample. The insertion unit 2 is fixed inside the turbine.

Each aspect of the present invention may include the following modifiedexample. The imaging device is a borescope.

In the first embodiment, the endoscope device 1 generates 3D dataindicating a 3D shape inside a turbine by using a feature regioncorresponding to a moving region without using a feature regioncorresponding to a stationary region. Therefore, the endoscope device 1can improve the reliability of processing of generating the 3D data.

A user can check the state of a rotor blade by checking an image of the3D data displayed on the display unit 5 or executing measurement usingthe 3D data. Therefore, the quality and the efficiency of an inspectionare improved.

First Modified Example of First Embodiment

A first modified example of the first embodiment of the presentinvention will be described. In the first embodiment described above,the endoscope device 1 classifies a region of a subject into a movingregion or a stationary region by using a moving amount of the regionbetween two or more images. In the first modified example of the firstembodiment, the endoscope device 1 classifies a region of a subject intoa moving region or a stationary region by using a luminance differenceof an image.

Processing executed by the endoscope device 1 will be described by usingFIG. 16 . FIG. 16 shows a procedure of 3D data generation processingexecuted by the endoscope device 1. The same processing as that shown inFIG. 7 will not be described.

After Step S105, the region determination unit 183 calculates aluminance difference between an image IMG(n−1) and an image IMGn (StepS120).

The region determination unit 183 executes the following processing inStep S120. The region determination unit 183 calculates a differencevalue between the luminance of each pixel in the image IMG(n−1) and theluminance of each pixel in the image IMGn. The region determination unit183 uses the luminance of the same pixel in the image IMG(n−1) and theimage IMGn.

In a case in which the imaging device 28 generates a color image havingthree channels, the region determination unit 183 may convert the imageIMG(n−1) and the image IMGn into gray scale images. Alternatively, theregion determination unit 183 may calculate three difference values byusing values of three channels and may calculate a statistic of thethree difference values as a luminance difference.

After Step S120, the region determination unit 183 determines whether afeature region of the image IMGn is a moving region or a stationaryregion based on the luminance difference (Step S107 a). After Step S107a, Step S108 is executed.

The region determination unit 183 executes the following processing inStep S107 a. The region determination unit 183 compares the luminancedifference of each pixel with a threshold value. When the luminancedifference is greater than the threshold value, the region determinationunit 183 determines that the pixel is included in a moving region. Whenthe luminance difference is less than or equal to the threshold value,the region determination unit 183 determines that the pixel is includedin a stationary region. The region determination unit 183 executes theabove-described processing for all the pixels of the image IMGn.

When the distal end 20 of the insertion unit 2 is moving, the entireregion seen in an image moves in the same direction. Therefore, theregion determination unit 183 may cause the position of the imageIMG(n−1) and the position of the image IMGn to match each other beforecalculating the luminance difference.

Details of Step S107 a and Step S108 will be described by using FIG. 17. Two or more feature regions in an image IMG(n−1) and two or morefeature regions in an image IMGn are shown in FIG. 17 . Each of thefeature regions is the same as that shown in FIG. 9 .

The region determination unit 183 calculates a luminance difference ofeach pixel of the image IMG(n−1) by using an image IMG(n−2) and theimage IMG(n−1). The region determination unit 183 classifies two or morefeature regions of the image IMG(n−1) into a moving region and astationary region MS(n−1) based on the luminance difference. The movingregion includes feature regions P2(n−1), P3(n−1), P4(n−1), and P5(n−1).

The region determination unit 183 calculates a luminance difference ofeach pixel of the image IMGn by using the image IMG(n−1) and the imageIMGn. The region determination unit 183 classifies two or more featureregions of the image IMGn into a moving region and a stationary regionMSn based on the luminance difference. The moving region includesfeature regions P2 n, P3 n, P4 n, and P5 n.

The 3D restoration unit 184 does not extract a feature regioncorresponding to the stationary region MS(n−1) in Step S108. Inaddition, the 3D restoration unit 184 does not extract a feature regioncorresponding to the stationary region MSn in Step S108.

Similar processing to that described above is executed each time a newimage is acquired in Step S102.

The order of Step S105, Step S120, and Step S107 a is not limited tothat shown in FIG. 16 . A modified example of processing executed by theendoscope device 1 will be described by using FIG. 18 . FIG. 18 shows aprocedure of 3D data generation processing executed by the endoscopedevice 1. The same processing as that shown in FIG. 16 will not bedescribed.

When the control unit 180 determines that the number n is not 1 in StepS104, the region determination unit 183 calculates a luminancedifference between an image IMG(n−1) and an image IMGn in Step S120.After Step S120, the region determination unit 183 determines whether afeature region of the image IMGn is a moving region or a stationaryregion based on the luminance difference in Step S107 a. At this time, afeature region of the image IMGn and a feature region of the imageIMG(n−1) have not been associated with each other.

After Step S107 a, the region detection unit 182 detects the samefeature regions (correspondence regions) in the image IMG(n−1) and theimage IMGn. At this time, the region detection unit 182 uses informationof feature regions determined to be moving regions in Step S107 a butdoes not use information of feature regions determined to be stationaryregions in Step S107 a (Step S105 a). After Step S105 a, Step S108 isexecuted.

Each aspect of the present invention may include the following modifiedexample. The CPU 18 determines whether at least part of a region in eachof two or more images of a component inside a turbine is a change region(moving region) or a non-change region (stationary region) based on thedifference of a pixel value between at least two images included in thetwo or more images in the region determination step (Step S107 a).

Each aspect of the present invention may include the following modifiedexample. After the CPU 18 determines whether the at least part of theregion in the two or more images is the change region or the non-changeregion in the region determination step (Step S107 a), the CPU 18detects two or more correspondence regions by using the change regionwithout using the non-change region in the region detection step (StepS105 a).

In the first modified example of the first embodiment, the endoscopedevice 1 can determine a moving region and a stationary region by usinga luminance difference of an image.

Second Modified Example of First Embodiment

A second modified example of the first embodiment of the presentinvention will be described. In the first embodiment and the firstmodified example of the first embodiment described above, the endoscopedevice 1 classifies a region of a subject into a moving region or astationary region by using two or more images. In the second modifiedexample of the first embodiment, the endoscope device 1 classifies aregion of a subject into a moving region or a stationary region by usingone image.

Processing executed by the endoscope device 1 will be described by usingFIG. 19 . FIG. 19 shows a procedure of 3D data generation processingexecuted by the endoscope device 1. The same processing as that shown inFIG. 7 will not be described.

After Step S105, the region determination unit 183 applies an imagerecognition technique to an image IMGn and determines a region of asubject in the image IMGn (Step S121).

The region determination unit 183 executes the following processing inStep S121. The region determination unit 183 processes the image IMGnand detects a rotor blade, a stator blade, a shroud, or the like seen inthe image IMGn. The region determination unit 183 may use known machinelearning as the image recognition technique. The region determinationunit 183 may use a method that does not use the machine learning. Amethod of determining a region in an image may be any method.

An example in which the region determination unit 183 uses the machinelearning will be described. A user gives a name of an object to aspecific region in an image acquired in a previously performedinspection. The image is a still image or a frame of a video. Before theprocessing shown in FIG. 19 is executed, the region determination unit183 learns the name given by the user as answer data. The answer dataare also called teacher data. The region determination unit 183 analyzesfeatures of an image used for learning by using the answer data andgenerates a learned model. When an inspection of a rotor blade of aturbine is performed, the rotor blade rotates and a stator blade or ashroud is fixed. The region determination unit 183 analyzes featuresrelated to movement of each object.

An external device may execute the above-described processing and maygenerate a learned model. The external device is the PC 41, a cloudserver, or the like. The endoscope device 1 may acquire the learnedmodel from the external device.

After the learned model is generated, the region determination unit 183determines an object seen in an image used for evaluation by using thelearned model in Step S121.

An example in which the region determination unit 183 does not use themachine learning will be described. The region determination unit 183calculates a uniquely designed image feature. The region determinationunit 183 executes clustering using the image feature and determines anobject seen in an image. The region determination unit 183 can apply atechnique such as support vector machine to the clustering.

After Step S121, the region determination unit 183 refers to informationof the region detected in Step S121. The region determination unit 183determines a feature region of a rotor blade in the image IMGn as amoving region and determines a feature region of a stator blade or ashroud in the image IMGn as a stationary region (Step S107 b). AfterStep S107 b, Step S108 is executed.

Details of Step S121 and S107 b will be described by using FIG. 20A andFIG. 20B. FIG. 20A shows an image IMGn generated by the imaging device28. FIG. 20B shows a moving region and a stationary region in the imageIMGn.

The region determination unit 183 analyzes the image IMGn in Step S121and detects an object OBJ1, an object OBJ2, and an object OBJ3. A numberis given to each object in accordance with the features of each object.For example, a number 1 is given to the object OBJ1, a number 2 is givento the object OBJ2, and a number 3 is given to the object OBJ3. When thetypes of two or more objects are the same or similar to each other,similar numbers may be given to the two or more objects. In the exampleshown in FIG. 21 , a similar number 2-1, 2-2, or the like is given tothe object OBJ2. The region determination unit 183 refers to objectinformation and acquires a name of each object.

The object information includes the number of an object and the name ofthe object. FIG. 21 shows an example of the object information. Thenumber and the name are associated with each other in the objectinformation. The name indicates a name of an object having featurescorresponding to the number.

A stator blade is associated with the number 1 of the object OBJ1 in theobject information. Therefore, the region determination unit 183determines that the name of the object OBJ1 is the stator blade. A rotorblade is associated with the number 2 of the object OBJ2 in the objectinformation. Therefore, the region determination unit 183 determinesthat the name of the object OBJ2 is the rotor blade. A shroud isassociated with the number 3 of the object OBJ3 in the objectinformation. Therefore, the region determination unit 183 determinesthat the name of the object OBJ3 is the shroud.

The region determination unit 183 determines that the feature region ofthe object OBJ2 is a moving region in Step S107 b. In addition, theregion determination unit 183 determines that the feature region of theobject OBJ1 and the feature region of the object OBJ3 are stationaryregions in Step S107 b.

Step S121 may be executed at any timing between a timing at which StepS102 is executed and a timing at which Step S107 b is executed.

The order of Step S105. Step S121, and Step S107 b is not limited tothat shown in FIG. 19 . A modified example of processing executed by theendoscope device 1 will be described by using FIG. 22 . FIG. 22 shows aprocedure of 3D data generation processing executed by the endoscopedevice 1. The same processing as that shown in FIG. 19 will not bedescribed.

When the control unit 180 determines that the number n is not 1 in StepS104, the region determination unit 183 determines a region of a subjectin an image IMGn in Step S121. After Step S121, the region determinationunit 183 determines a feature region of a rotor state in the image IMGnas a moving region in Step S107 b. In addition, the region determinationunit 183 determines a feature region of a stator blade or a shroud inthe image IMGn as a stationary region in Step S107 b. At this time, afeature region of the image IMGn and a feature region of an imageIMG(n−1) have not been associated with each other.

After Step S107 b, the region detection unit 182 detects the samefeature regions (correspondence regions) in the image IMG(n−1) and theimage IMGn. At this time, the region detection unit 182 uses informationof feature regions determined to be moving regions in Step S107 b butdoes not use information of feature regions determined to be stationaryregions in Step S107 b (Step S105 b). After Step S105 b, Step S108 isexecuted.

Each aspect of the present invention may include the following modifiedexample. The CPU 18 determines a subject seen in one image included intwo or more images, thus determining whether at least part of a regionin each of the two or more images is a change region (moving region) ora non-change region (stationary region) in the region determination step(Step S107 b).

Each aspect of the present invention may include the following modifiedexample. After the CPU 18 determines whether the at least part of theregion in the two or more images is the change region or the non-changeregion in the region determination step (Step S107 b), the CPU 18detects two or more correspondence regions by using the change regionwithout using the non-change region in the region detection step (StepS105 b).

In the second modified example of the first embodiment, the endoscopedevice 1 can determine a moving region and a stationary region bydetermining a type of subject in an image.

Second Embodiment

A second embodiment of the present invention will be described. When aninspection is being performed, the endoscope device 1 generates an imageand generates 3D data of a subject. In other words, the endoscope device1 simultaneously executes generation of an image and generation of 3Ddata. In order to avoid a failure in the 3D restoration processing andfacilitate an efficient inspection, the endoscope device 1 offersvarious assistance functions.

The CPU 18 shown in FIG. 6 is changed to a CPU 18 a shown in FIG. 23 .FIG. 23 shows a configuration of the CPU 18 a. The same configuration asthat shown in FIG. 6 will not be described.

The CPU 18 a functions as a control unit 180, an image acquisition unit181, a region detection unit 182, a region determination unit 183, a 3Drestoration unit 184, a display control unit 185, a state determinationunit 186, and a notification unit 187. At least one of the blocks shownin FIG. 23 may be constituted by a different circuit from the CPU 18 a.

Each unit of the CPU 18 a may be constituted by at least one of aprocessor and a logic circuit. Each unit of the CPU 18 a may include oneor a plurality of processors. Each unit of the CPU 18 a may include oneor a plurality of logic circuits.

The state determination unit 186 determines a state of an inspection.For example, the state of the inspection is indicated by the quality ofan image generated by the imaging device 28, the area of a stationaryregion seen in the image, movement of the camera, or the progress ofobservation of a subject. The state determination unit 186 generatesinspection state information regarding the state of the inspection.

The notification unit 187 executes notification processing of notifyinga user of the inspection state information. For example, thenotification unit 187 generates a graphic image signal corresponding tothe inspection state information. The notification unit 187 outputs thegraphic image signal to the video-signal-processing circuit 12. Similarprocessing to that described above is executed, and the display unit 5displays a message including the inspection state information. By doingthis, the notification unit 187 displays the inspection stateinformation on the display unit 5.

The notification unit 187 may output sound data to a speaker and maycause the speaker to generate a sound corresponding to the inspectionstate information. The notification unit 187 may output a control signalindicating a pattern of vibration to a vibration generator and may causethe vibration generator to generate vibration having the patterncorresponding to the inspection state information. The notification unit187 may output a control signal indicating a pattern of light emissionto a light source and may cause the light source to generate lighthaving the pattern corresponding to the inspection state information.

Hereinafter, an example in which the display unit 5 displays a messageincluding the inspection state information will be described. In thiscase, the display control unit 185 may function as the notification unit187.

A first example of processing executed by the endoscope device 1 will bedescribed by using FIG. 24 and FIG. 25 . FIG. 24 and FIG. 25 show aprocedure of 3D data generation processing executed by the endoscopedevice 1. The same processing as that shown in FIG. 7 will not bedescribed.

For example, a user inputs an instruction to start the 3D datageneration processing into the endoscope device 1 by operating theoperation unit 4. In a case in which the display unit 5 is constitutedas a touch panel, the user inputs the instruction into the endoscopedevice 1 by touching the display of the display unit 5. The control unit180 accepts the instruction and starts the 3D data generationprocessing.

The control unit 180 may start the 3D data generation processing when asubject seen in an image generated by the imaging device 28 isdetermined to be stationary for a predetermined period of time.Alternatively, the control unit 180 may start the 3D data generationprocessing when the composition of photography for acquiring the imagematches a predetermined composition. For example, the control unit 180may check whether the present composition matches the previouscomposition by using an inspection image acquired in photographyperformed in a previous inspection. A user may perform work to match thecomposition of photography with the predetermined composition or tocheck the composition of photography by manually performing anoperation. When the 3D data generation processing is started, thecontrol unit 180 may transmit control information to the turning tool 43and may cause the turning tool 43 to start rotation.

After Step S102, the state determination unit 186 determines the qualityof an image IMGn (Step S130).

For example, when halation has occurred in the image IMGn or the imageIMGn is dark, it is difficult to execute the 3D restoration processing.Also, when there are few patterns on a subject seen in the image IMGn,it is difficult to execute the 3D restoration processing. Also, whenblades rotate fast, it is difficult to execute the 3D restorationprocessing due to the motion blur. When these factors occur, the statedetermination unit 186 determines that the quality of the image IMGn islow. When these factors do not occur, the state determination unit 186determines that the quality of the image IMGn is high.

When the state determination unit 186 determines that the quality of theimage IMGn is high in Step S130. Step S103 is executed. When the statedetermination unit 186 determines that the quality of the image IMGn islow in Step S130, the state determination unit 186 generates inspectionstate information indicating that the quality of the image is low or itis difficult to execute the 3D restoration processing. The notificationunit 187 executes the notification processing and displays theinspection state information on the display unit 5 (Step S133). Theinspection state information functions as an alert. The inspection stateinformation may include information that encourages a user to change thecomposition of photography, a setting condition of an image, or thelike.

When Step S133 is executed, the control unit 180 may transmit controlinformation to stop the turning tool 43 to the turning tool 43. Theturning tool 43 may stop rotation of rotor blades based on the controlinformation. When Step S133 is executed, the 3D data generationprocessing is completed.

After Step S106, the control unit 180 adjusts the rotation speed ofrotor blades based on the moving amount calculated in Step S106 (StepS131). After Step S131, Step S107 is executed.

The control unit 180 executes the following processing in Step S131. Thecontrol unit 180 calculates the difference between the moving amountcalculated in Step S106 and a target amount of movement. The targetamount of movement is set in advance. When the difference is greaterthan a predetermined value, the control unit 180 transmits controlinformation to reduce the rotation speed of the rotor blades to theturning tool 43. The turning tool 43 reduces the rotation speed of therotor blades based on the control information.

When the distal end 20 of the insertion unit 2 is near a subject, thesubject moves fast in an image generated by the imaging device 28.Therefore, the endoscope device 1 needs to reduce the rotation speed ofthe rotor blades. Since the rotation speed of the rotor blades iscontrolled based on the calculated moving amount, the endoscope device 1can suppress an influence on an inspection caused by the composition inobservation.

After Step S106, Step S107 may be executed without executing Step S131.

After Step S107, the state determination unit 186 determines whether thearea of a stationary region in the image IMGn is large (Step S132).

The state determination unit 186 executes the following processing inStep S132. For example, the state determination unit 186 calculates thenumber of pixels of a stationary region in the image IMGn. The statedetermination unit 186 compares the calculated number of pixels with apredetermined value. When the number of pixels is larger than thepredetermined value, the state determination unit 186 determines thatthe area of a stationary region is large. When the number of pixels isless than or equal to the predetermined value, the state determinationunit 186 determines that the area of a stationary region is small. Thepredetermined value is set in advance. The predetermined value may bevariable.

An example of Step S132 will be described by using FIG. 26A and FIG.26B. FIG. 26A shows an image IMGn generated by the imaging device 28.FIG. 26B shows a moving region and a stationary region in the imageIMGn.

The image IMGn includes a stationary region RG20, a stationary regionRG21, and a moving region RG22. The ratio of the moving region RG22 tothe entire image IMGn is small. In this case, it is highly probable thatthe endoscope device 1 fails in the 3D restoration processing.

When the state determination unit 186 determines that the area of astationary region is small in Step S132, Step S108 is executed. When thestate determination unit 186 determines that the area of a stationaryregion is large in Step S132, the state determination unit 186generates, in Step S133, inspection state information indicating that itis difficult to execute the 3D restoration processing. The notificationunit 187 executes the notification processing and displays theinspection state information on the display unit 5 in Step S133. Theinspection state information functions as an alert. The inspection stateinformation may include information that encourages a user to change thecomposition of photography.

When the number n is 2, Step S132 is executed for the first time. Atthis time, Step S109 has not been executed yet. When the statedetermination unit 186 determines that the area of the stationary regionis large in Step S132, Step S133 is executed as described above and the3D data generation processing is completed. At this time, the controlunit 180 may cause the turning tool 43 to stop rotation of the rotorblades.

On the other hand, after Step S132 is executed once or more, Step S132may be executed again and the state determination unit 186 may determinethat the area of the stationary region is large. This means a situationin which the stationary region expands in the process of the 3D datageneration processing. When a user stops the rotation of the rotorblades in order to change the position of the distal end 20 of theinsertion unit 2, the stationary region expands. A condition under whichthe rotation of the rotor blades stops is not limited to the generationof an instruction from a user. For example, the endoscope device 1 mayautomatically stop the rotation of the rotor blades. A condition underwhich the rotation of the rotor blades stops is not limited to theabove-described examples.

A situation in which the stationary region expands will be described.There is a case in which an inspection of a large rotor blade isperformed. Such a large rotor blade is disposed in a compressor sectionor a turbine section on a low-pressure side in many cases. Only part ofthe rotor blade comes in the visual field of the camera. In thisinspection, the position of the distal end 20 is changed and photographyof the rotor blade is performed twice or more.

Details of an inspection of a large rotor blade will be described byusing FIG. 27A, FIG. 27B, FIG. 28A, and FIG. 28B. Each drawing shows anobservation position of eight rotor blades RT20.

First, the distal end 20 is fixed at a position shown in FIG. 27A. Atthis time, illumination light LT20 is emitted to the upper part of therotor blade RT20 as shown in FIG. 27B. The rotor blades rotate once in adirection DR20 in this state. At this time, the rotor blades rotate by360 degrees or more. While the rotor blades rotate, Steps S101 to S110are repeatedly executed.

After the rotor blades rotate once, the rotation of the rotor blades isstopped. The position of the distal end 20 is changed, and the distalend 20 is fixed at a position shown in FIG. 28A. At this time,illumination light LT20 is emitted to the lower part of the rotor bladeRT20 as shown in FIG. 28B. The rotor blades rotate once in a directionDR20 in this state, and Steps S101 to S110 are repeatedly executed.

When the rotation of the rotor blades is stopped in order to change theposition of the distal end 20, the entire subject seen in an imagegenerated by the imaging device 28 comes to a standstill. Therefore, thestate determination unit 186 determines that the area of a stationaryregion is large in Step S132. In this case, the endoscope device 1 maycontinue the 3D data generation processing. In other words, after StepS133 is executed, Step S110 may be executed. In this case, the endoscopedevice 1 stops Step S108 and Step S109 and continues Steps S101 to S107.

While the distal end 20 moves from the position shown in FIG. 27A to theposition shown in FIG. 28A, the entire subject seen in an imagegenerated by the imaging device 28 moves. Therefore, the statedetermination unit 186 determines that the area of a stationary regionis small in Step S132. In this case, the endoscope device 1 restartsStep S108 and Step S109 and continues Steps S101 to S110.

The distal end 20 reaches the position shown in FIG. 28A and is fixed.At this time, the entire subject seen in an image generated by theimaging device 28 comes to a standstill. Therefore, the statedetermination unit 186 determines that the area of a stationary regionis large in Step S132. In this case, the endoscope device 1 may stopStep S108 and Step S109 and may continue Steps S101 to S107.

Thereafter, the rotation of the rotor blades is restarted. Therefore,the state determination unit 186 determines that the area of astationary region is small in Step S132. In this case, the endoscopedevice 1 restarts Step S108 and Step S109 and continues Steps S101 toS110.

In an inspection of a large rotor blade, Steps S101 to S107 arerepeatedly executed even when the rotation of the rotor blades isstopped. Due to this, the endoscope device 1 accumulates two or moreimages generated by the imaging device 28 and information of the samefeature regions in the two or more images. When the rotation of therotor blades is restarted and the 3D restoration processing in Step S109is executed again, the 3D restoration unit 184 can restore a 3D shape ofa subject by using the accumulated images and information. Therefore,the endoscope device 1 can generate a piece of 3D data indicating theentire 3D shapes of two or more rotor blades.

When the number n is two or more and the state determination unit 186determines that the area of a stationary region is large in Step S132,the state determination unit 186 may generate inspection stateinformation indicating that the rotation of the rotor blades is stoppedin Step S133.

The state determination unit 186 may receive state informationindicating a driving state of the turning tool 43 from the turning tool43. The state determination unit 186 may monitor the state of the rotorblades based on the state information. The state information indicateswhether the turning tool 43 is rotating the rotor blades. The statedetermination unit 186 may detect that the rotation of the rotor bladeis stopped based on the state information and may generate inspectionstate information indicating that the rotation of the rotor blades isstopped in Step S133.

When an inspection of the entire regions of the rotor blades iscompleted and the state determination unit 186 determines that the areaof a stationary region is large in Step S132, the endoscope device 1 mayexecute Step S133 and may complete the 3D data generation processing.The endoscope device 1 may use a method of detecting that a subjectobserved in an inspection in progress has been observed again as amethod of detecting that the inspection of the entire regions of therotor blades is completed. Such a method will be described later in athird example.

A user may input information indicating that the inspection of theentire regions of the rotor blades is completed into the endoscopedevice 1 by operating the operation unit 4. In a case in which thedisplay unit 5 is constituted as a touch panel, the user may input theinformation into the endoscope device 1 by touching the display of thedisplay unit 5. The state determination unit 186 may detect that theinspection is completed based on the information.

A second example of processing executed by the endoscope device 1 willbe described by using FIG. 29 and FIG. 30 . FIG. 29 and FIG. 30 show aprocedure of 3D data generation processing executed by the endoscopedevice 1. The same processing as that shown in FIG. 7 will not bedescribed.

After Step S106. Step S131 is executed. Step S131 shown in FIG. 30 isthe same as that shown in FIG. 25 . After Step S131, Step S107 isexecuted. After Step S106, Step S107 may be executed without executingStep S131.

After Step S107, the state determination unit 186 determines movement ofthe camera (Step S134).

The state determination unit 186 executes the following processing inStep S134. For example, the state determination unit 186 determines themovement of the camera by analyzing an image IMGn. In a case in which aninertial measurement unit (IMU) that determines an acceleration and anangular velocity of the distal end 20 of the insertion unit 2 isdisposed in the distal end 20, the state determination unit 186 maydetermine the movement of the camera based on a value measured by theIMU. When the amount of the movement of the camera is very small, thestate determination unit 186 may determine that the camera is notmoving.

When the state determination unit 186 determines that the camera is notmoving in Step S134, Step S108 is executed. When the state determinationunit 186 determines that the camera is moving in Step S134, the statedetermination unit 186 generates inspection state information indicatingthat the camera is moving. The notification unit 187 executes thenotification processing and displays the inspection state information onthe display unit 5 (Step S135). The inspection state informationfunctions as an alert.

When Step S135 is executed, the control unit 180 may transmit controlinformation to stop the turning tool 43 to the turning tool 43. Theturning tool 43 may stop rotation of rotor blades based on the controlinformation. When Step S135 is executed, the 3D data generationprocessing is completed.

As described above, there is a case in which the position of the distalend 20 is changed and photography of a rotor blade is performed twice ormore in order to perform an inspection of a large rotor blade. When thedistal end 20 is moving, the state determination unit 186 determinesthat the camera is moving in Step S134. In this case, the endoscopedevice 1 may continue the 3D data generation processing. In other words,after Step S135 is executed, Step S110 may be executed. In this case,the endoscope device 1 stops Step S108 and Step S109 and continues StepsS101 to S107.

When an inspection of the entire regions of the rotor blades iscompleted and the state determination unit 186 determines that thecamera is moving in Step S134, the endoscope device 1 may execute StepS135 and may complete the 3D data generation processing. The endoscopedevice 1 may use a method of detecting that a subject has been observedagain in an inspection in progress as a method of detecting that theinspection of the entire regions of the rotor blades is completed. Sucha method will be described later in a third example.

As described above, a user may input information indicating that theinspection of the entire regions of the rotor blades is completed intothe endoscope device 1 by operating the operation unit 4 or the touchpanel. The state determination unit 186 may detect that the inspectionis completed based on the information.

A third example of processing executed by the endoscope device 1 will bedescribed by using FIG. 29 and FIG. 31 . FIG. 29 and FIG. 31 show aprocedure of 3D data generation processing executed by the endoscopedevice 1. The same processing as that shown in FIG. 7 will not bedescribed.

After Step S106, Step S131 is executed. Step S131 shown in FIG. 31 isthe same as that shown in FIG. 25 . After Step S131, Step S107 isexecuted. After Step S106, Step S107 may be executed without executingStep S131.

After Step S109, the state determination unit 186 determines whether asubject has been observed again in an inspection in progress (StepS136). When rotor blades have rotated once, a subject having alreadybeen observed is observed again.

The state determination unit 186 executes the following processing inStep S136. For example, the state determination unit 186 receives stateinformation indicating a driving state of the turning tool 43 from theturning tool 43. The state determination unit 186 monitors the state ofrotor blades based on the state information. The state informationindicates the rotation angle of the rotor blades. When the rotationangle of the rotor blades is less than 360 degrees, the statedetermination unit 186 determines that the subject has not been observedagain. When the rotation angle of the rotor blades is greater than orequal to 360 degrees, the state determination unit 186 determines thatthe subject has been observed again.

Alternatively, the state determination unit 186 analyzes an imagegenerated by the imaging device 28 and determines whether a subjecthaving a similar feature to that of a subject having already beenobserved is seen in the image. When the subject having the feature isnot seen in the image, the state determination unit 186 determines thatthe subject has not been observed again. When the subject having thefeature is seen in the image, the state determination unit 186determines that the subject has been observed again.

Alternatively, the state determination unit 186 analyzes an imagegenerated by the imaging device 28 and recognizes a rotor blade seen inthe image. The state determination unit 186 counts the number ofrecognized rotor blades. The number (number in design) of rotor bladesfixed on the circumference of the disk is known. The state determinationunit 186 determines whether the number of recognized rotor blades hasreached the number in design. When the number of recognized rotor bladeshas not reached the number in design, the state determination unit 186determines that the subject has not been observed again. When the numberof recognized rotor blades has reached the number in design, the statedetermination unit 186 determines that the subject has been observedagain.

When the state determination unit 186 determines that the subject hasnot been observed again in Step S136, Step S110 is executed. When thestate determination unit 186 determines that the subject has beenobserved again in Step S136, the state determination unit 186 generatesinspection state information indicating that the subject has beenobserved again. The notification unit 187 executes the notificationprocessing and displays the inspection state information on the displayunit 5 (Step S137). The inspection state information functions as analert. The inspection state information may include information thatencourages a user to complete the inspection. As described above, in acase in which the position of the distal end 20 is changed andphotography of a rotor blade is performed twice or more in order toperform an inspection of a large rotor blade, the inspection stateinformation may include information that encourages a user to change theposition of the distal end 20.

When Step S137 is executed, the control unit 180 may transmit controlinformation to stop the turning tool 43 to the turning tool 43. Theturning tool 43 may stop rotation of rotor blades based on the controlinformation. When Step S137 is executed, the 3D data generationprocessing is completed.

In the first to third examples, Step S106 and Step S107 may be changedto Step S120 and Step S107 a shown in FIG. 16 , respectively. In thefirst to third examples, Step S106 and Step S107 may be changed to StepS121 and Step S107 b shown in FIG. 19 , respectively.

Each aspect of the present invention may include the following modifiedexample. The CPU 18 determines the position of the insertion unit 2inside a turbine in a position determination step (Step S134). When theposition of the insertion unit 2 has changed, the CPU 18 executesprocessing of notifying a user of information indicating a change of theposition of the insertion unit 2 in a notification step (Step S135).

Each aspect of the present invention may include the following modifiedexample. Before the CPU 18 executes the data generation step (Step S109)for the first time, the CPU 18 calculates the area of a non-changeregion (stationary region) in an image included in two or more images ofa component inside a turbine in a calculation step (Step S132). When thearea is larger than a predetermined value, the CPU 18 executesprocessing of notifying a user of an alert in a notification step (StepS133).

Each aspect of the present invention may include the following modifiedexample. A first object (rotor blade) rotates inside the turbine due toa driving force generated by the turning tool 43 (driving device). TheCPU 18 determines whether the first object has rotated once inside theturbine in a rotation determination step (Step S136). When the CPU 18determines that the first object has rotated once inside the turbine,the CPU 18 executes processing of notifying a user of informationindicating that the first object has rotated once inside the turbine ina notification step (Step S137).

Each aspect of the present invention may include the following modifiedexample. A first object (rotor blade) rotates inside the turbine due toa driving force generated by the turning tool 43 (driving device). Whilethe first object rotates, the CPU 18 repeatedly executes the regiondetection step (Step S105), the region determination step (Step S107),and the data generation step (Step S109). When the rotation of the firstobject is stopped, the CPU 18 stops execution of the data generationstep and continues execution of the region detection step and the regiondetermination step.

Each aspect of the present invention may include the following modifiedexample. When the first object (rotor blade) starts to rotate again, theCPU 18 restarts the data generation step (Step S109).

Each aspect of the present invention may include the following modifiedexample. The CPU 18 determines a rotation state of the first object(rotor blade) by using an image included in two or more images of acomponent inside a turbine.

Each aspect of the present invention may include the following modifiedexample. The CPU 18 determines a rotation state of the first object(rotor blade) by monitoring the state of the turning tool 43 (drivingdevice).

In the second embodiment, the endoscope device 1 notifies a user ofinspection state information regarding the state of an inspection.Therefore, the endoscope device 1 can avoid the failure in the 3Drestoration processing and can facilitate an efficient inspection.

Third Embodiment

A third embodiment of the present invention will be described. In thethird embodiment, the endoscope device 1 shown in FIG. 1 and FIG. 2 isused. In the first embodiment and the second embodiment, the endoscopedevice 1 uses a single-eye optical adaptor. On the other hand, in thethird embodiment, the endoscope device 1 uses a stereo optical adaptor,which is an example of an optical adaptor used for measurement.

The endoscope device 1 uses a stereo optical adaptor 30 shown in FIG. 32and FIG. 33 . FIG. 32 and FIG. 33 show a configuration of the distal end20 of the insertion unit 2 and the stereo optical adaptor 30. FIG. 32shows an external appearance of the distal end 20 and the stereo opticaladaptor 30. FIG. 33 shows a cross-section of the distal end 20 and thestereo optical adaptor 30. A first illumination optical system 51, asecond illumination optical system 52, a first objective optical system53, and a second objective optical system 54 are disposed in the distalend of the stereo optical adaptor 30. A cross-section passing throughthe first objective optical system 53 and the second objective opticalsystem 54 is shown in FIG. 33 .

The stereo optical adaptor 30 is mounted on the distal end 20 of theinsertion unit 2. The stereo optical adaptor 30 includes a fixed ring 50on which a female screw 50 a is formed. A male screw 20 a is formed onthe distal end 20 of the insertion unit 2. The stereo optical adaptor 30is screwed together with the male screw 20 a by using the female screw50 a and is fixed on the distal end 20.

The imaging device 28 is disposed in the distal end 20. The firstobjective optical system 53 and the second objective optical system 54form two optical images on the imaging device 28. The imaging device 28converts the two optical images into an imaging signal. A signal line 2b is connected to the imaging device 28. The imaging signal is providedto the CCU 9 via the signal line 2 b and the endoscope unit 8. The CCU 9converts the imaging signal into a video signal and provides the videosignal to the video-signal-processing circuit 12.

The first objective optical system 53 forms a first optical image of asubject seen from a first viewpoint. The second objective optical system54 forms a second optical image of the subject seen from a secondviewpoint different from the first viewpoint. The imaging device 28includes an effective region on which the first optical image and thesecond optical image are formed. For example, the first optical image isformed on the left region in the effective region, and the secondoptical image is formed on the right region in the effective region.

The imaging device 28 forms a stereo image corresponding to the firstoptical image and the second optical image. The stereo image includes apair of two images. In other words, the stereo image includes an imageof the subject seen from the first viewpoint and an image of the subjectseen from the second viewpoint.

Processing executed by the endoscope device 1 will be described by usingFIG. 34 . FIG. 34 shows a procedure of 3D data generation processingexecuted by the endoscope device 1. The same processing as that shown inFIG. 7 will not be described.

When the control unit 180 determines that the number n is 1 in StepS104, the 3D restoration unit 184 executes initial 3D restorationprocessing (Step S140). After Step S140. Step S101 is executed.Hereinafter, details of Step S140 will be described.

A method of calculating three-dimensional coordinates (3D coordinates)of a point of interest in stereo measurement will be described byreferring to FIG. 35 . The center of a line segment connecting a leftoptical center (first optical center 63) and a right optical center(second optical center 64) is defined as an origin O. In addition, anx-axis, a y-axis, and a z-axis shown in FIG. 35 are defined.

An image including a subject image is used. The subject image isobtained via a left optical system and a right optical system. As thefollowing Expressions (4) to (6) show, the 3D coordinates (X, Y, Z) of apoint of interest 60 are calculated by using a principle oftriangulation. The two-dimensional coordinates (2D coordinates) of apoint of interest 61 and the 2D coordinates of a point of interest 62are (X_(L), Y_(L)) and (X_(R), Y_(R)), respectively. The point ofinterest 61 is on a left image surface on which distortion correction isperformed. The point of interest 62 is on a right image surface on whichthe distortion correction is performed.

The origin of the point of interest 61 is an intersection point O_(L),and the origin of the point of interest 62 is an intersection pointO_(R). The intersection point O_(L) is at a position at which theoptical axis of the left optical system and the image surface intersecteach other. The intersection point O_(R) is at a position at which theoptical axis of the right optical system and the image surface intersecteach other. The distance between the first optical center 63 and thesecond optical center 64 is D. A parameter F indicates a focal length. Aparameter t is expressed as D/(X_(R)−X_(L)).

X=t×X _(R) +D/2  (4)

Y=·t×Y _(R)  (5)

Z=t×F  (6)

In a case in which the coordinates of each of the points 61 and 62 ofinterest are determined as described above, the CPU 18 can calculate the3D coordinates of the point of interest 60 by using the parameter D andthe parameter F. The parameter D and the parameter F are calculated atthe time of factory shipment of the stereo optical adaptor 30.Alternatively, the parameter D and the parameter F are calculated in aprocess such as setting-up of the endoscope device 1 before aninspection is performed. The 3D restoration unit 184 can restore a 3Dshape of a subject by executing the above-described processing for allpixels of a stereo image.

After Step S108, the 3D restoration unit 184 executes the 3D restorationprocessing by using the feature region corresponding to the movingregion (Step S109 d). After Step S109 d, Step S110 is executed.

The 3D restoration unit 184 uses a stereo image acquired by the imagingdevice 28 in Step S109 d. The 3D restoration processing in Step S109 dis similar to that shown in FIG. 13 . When the stereo optical adaptor 30is mounted on the distal end 20, the 3D restoration unit 184 can use apair of a left image and a right image included in one stereo image inaddition to two or more stereo images acquired at different time points.

When a single-eye optical adaptor is mounted on the distal end 20 in thefirst embodiment, the 3D restoration unit 184 uses a first image and asecond image acquired at, for example, different time points andexecutes the 3D restoration processing by following epipolar constraintof the images.

On the other hand, when the stereo optical adaptor 30 is mounted on thedistal end 20 in the third embodiment, the 3D restoration unit 184 usesa first stereo image acquired at a first time point and a second stereoimage acquired at a second time point and executes the 3D restorationprocessing by following the epipolar constraint of the stereo images.When the stereo optical adaptor 30 is mounted on the distal end 20, theepipolar constraint with respect to a stereo image is defined inaccordance with a known external parameter. The external parameterindicates a positional relationship between two images that constitute astereo image. The external parameter is calculated at the time offactory shipment. Alternatively, the external parameter is calculated ina process such as setting-up of the endoscope device 1 before aninspection is performed.

Each stereo image includes a left image and a right image. Accordingly,the 3D restoration unit 184 can execute the 3D restoration processing byfollowing the epipolar constraint of, for example, the following fourpairs.

-   -   A first pair: a pair of a left image of a first stereo image and        a left image of a second stereo image.    -   A second pair: a pair of a right image of the first stereo image        and a right image of the second stereo image.    -   A third pair: a pair of the left image of the first stereo image        and the right image of the first stereo image.    -   A fourth pair: a pair of the left image of the second stereo        image and the right image of the second stereo image.

Since the above-described four pairs are used in Step S109 a and StepS109 b shown in FIG. 13 , the accuracy of the 3D restoration processingis improved and the stability of a restored 3D shape is improved.

For example, camera calibration is performed at the time of factoryshipment. Therefore, the positions and the postures of the camera whenthe camera acquires the above-described third and fourth pairs arecalculated in advance. Accordingly, the 3D restoration unit 184 canrestore a 3D shape of a subject with high accuracy.

In a case in which the stereo optical adaptor 30 is used, the cameracalibration has already been performed. Therefore, the 3D restorationunit 184 can restore a 3D shape having an absolute scale. The size ofthe 3D shape is the same as that of an actual subject. Therefore, StepS109 c shown in FIG. 13 can be omitted.

In the third embodiment, the endoscope device 1 acquires a stereo imageby using the stereo optical adaptor 30 and uses the stereo image in the3D restoration processing. Therefore, the endoscope device 1 can stablyexecute the 3D restoration processing with high accuracy, compared tothe first embodiment in which a single-eye optical adaptor is used. Thethird embodiment can be applied to the second embodiment in which theendoscope device 1 simultaneously generates an image and 3D data.

Fourth Embodiment

A fourth embodiment of the present invention will be described. In thefourth embodiment, the endoscope device 1 shown in FIG. 1 and FIG. 2 isused. In addition, the stereo optical adaptor 30 shown in FIG. 32 andFIG. 33 is used in the fourth embodiment.

The first objective optical system 53 and the second objective opticalsystem 54 form two optical images on the effective region of the imagingdevice 28 at the same time in the third embodiment. On the other hand,one of the first objective optical system 53 and the second objectiveoptical system 54 forms an optical image on the entire effective region,and then the other of the first objective optical system 53 and thesecond objective optical system 54 forms an optical image on the entireeffective region in the fourth embodiment. The endoscope device 1acquires a left image and a right image in a time-division manner in thefourth embodiment.

FIG. 36 shows a configuration of the distal end 20 of the insertion unit2 and the stereo optical adaptor 30. The lens 21 and the imaging device28 are disposed in the distal end 20. The imaging device 28 includes aneffective region 28 a. The stereo optical adaptor 30 includes a firstobjective optical system 53, a second objective optical system 54, andan optical-path-setting unit 55.

For example, the first objective optical system 53 and the secondobjective optical system 54 are a combination of a concave lens and aconvex lens. The second objective optical system 54 is disposed suchthat the second objective optical system 54 has parallax for the firstobjective optical system 53. In other words, the first objective opticalsystem 53 and the second objective optical system 54 are separated fromeach other in a parallax direction. The parallax direction is adirection of a straight line passing through the optical center of thefirst objective optical system 53 and the optical center of the secondobjective optical system 54. Light incident on the first objectiveoptical system 53 passes through a first optical path L1. Light incidenton the second objective optical system 54 passes through a secondoptical path L2 different from the first optical path L1. The firstobjective optical system 53 forms a first optical image of a subject,and the second objective optical system 54 forms a second optical imageof the subject.

The optical-path-setting unit 55 switches optical paths between thefirst optical path L1 and the second optical path L2 such that eitherthe first optical image or the second optical image is formed on theeffective region 28 a of the imaging device 28. By doing this, theoptical-path-setting unit 55 sets either the first optical path L1 orthe second optical path L2 as an imaging optical path. Theoptical-path-setting unit 55 is configured to transmit light passingthrough either the first optical path L1 or the second optical path L2and is configured to block light passing through the other.

For example, the optical-path-setting unit 55 includes a shutter to beinserted into only one of the first optical path L1 and the secondoptical path L2. When the optical-path-setting unit 55 transmits lightof the first optical path L1, the shutter is inserted into the secondoptical path 12 and light of the second optical path L2 is blocked. Whenthe optical-path-setting unit 55 transmits light of the second opticalpath L2, the shutter is inserted into the first optical path L1 andlight of the first optical path L1 is blocked. The control unit 180controls the operation of the optical-path-setting unit 55. The lens 21forms a subject image on the effective region 28 a of the imaging device28 based on either the light passing through the first optical path L1or the light passing through the second optical path L2.

The imaging device 28 includes the effective region 28 a on which thefirst optical image of the light passing through the first optical pathL1 and the second optical image of the light passing through the secondoptical path L2 are formed. The imaging device 28 transforms the firstoptical image into a left image at a first timing and transforms thesecond optical image into a right image at a second timing differentfrom the first timing. The left image and the right image constitute astereo image.

Processing executed by the endoscope device 1 will be described by usingFIG. 37 . FIG. 37 shows a procedure of 3D data generation processingexecuted by the endoscope device 1. The same processing as that shown inFIG. 7 will not be described.

The image acquisition unit 181 acquires the image IMGn from the RAM 14in Step S102. The image IMGn is a left image or a right image.

After Step S102, the control unit 180 acquires information indicatingthe type of the image IMGn acquired in Step S102 from the RAM 14 (StepS150). The information indicates the left image or the right image. Thecontrol unit 180 generates the information indicating the type of theimage TMGn based on a control signal used for setting the imagingoptical path. Alternatively, the control unit 180 generates theinformation indicating the type of the image IMGn based on a result ofimage processing that uses the parallax between the first objectiveoptical system 53 and the second objective optical system 54. Theinformation indicating the type of the image IMGn is associated with thenumber n and is stored on the RAM 14 in advance.

After Step S108, the 3D restoration unit 184 determines whether theimage IMG(n−1) and the image IMGn constitute a stereo image (Step S151).

In the following descriptions, the left image is expressed as I_(L), andthe right image is expressed as I_(R). For example, the left image andthe right image are alternately acquired as follows.

I _(L)(n=1),I _(R)(n=2),I _(L)(n=3),I _(R)(n=4), . . .

When the number n is 5, the image IMG(n−1) is the right image I_(R) andthe image IMGn is the left image I_(L). At this time, the 3D restorationunit 184 determines that the image IMG(n−1) and the image IMGnconstitute a stereo image.

When the 3D restoration unit 184 determines that the image IMG(n−1) andthe image IMGn constitute a stereo image in Step S151, the 3Drestoration unit 184 determines whether the distal end 20 is stationary(Step S152).

When the distal end 20 is moving, the entire region seen in the imageIMG(n−1) or the image IMGn moves. Therefore, the region determinationunit 183 determines that all the small regions of the image IMGn aremoving regions in Step S107. In this case, the 3D restoration unit 184determines that the distal end 20 is not stationary in Step S152.

When the distal end 20 is stationary, the region determination unit 183determines that at least some of the small regions of the image IMGn arestationary regions in Step S107. In this case, the 3D restoration unit184 determines that the distal end 20 is stationary in Step S152.

When the 3D restoration unit 184 determines that the distal end 20 isstationary in Step S152, the 3D restoration unit 184 uses a stereo imageand executes the 3D restoration processing by following the epipolarconstraint. This processing is similar to Step S140. At this time, the3D restoration unit 184 uses the feature region corresponding to themoving region (Step S153). After Step S153. Step S110 is executed.

When the 3D restoration unit 184 determines that the image IMG(n−1) andthe image IMGn do not constitute a stereo image in Step S151 ordetermines that the distal end 20 is not stationary in Step S152, the 3Drestoration unit 184 executes the 3D restoration processing withoutfollowing the epipolar constraint. At this time, the 3D restoration unit184 executes the 3D restoration processing by using the feature regioncorresponding to the moving region (Step S154). After Step S154, StepS110 is executed.

Step S154 is the same as Step S109 shown in FIG. 7 . The type of imagesused in the 3D restoration processing in Step S154 is different fromthat of images used in the 3D restoration processing in Step S109.

In the first embodiment, the imaging device 28 generates a single-eyeimage based on an optical image formed by a single-eye optical adaptor.The 3D restoration unit 184 executes the 3D restoration processing inStep S109 by using single-eye images regardless of the number n.

In the fourth embodiment, the 3D restoration unit 184 may execute the 3Drestoration processing by using only left images at all times. When theimage IMGn is a right image, the 3D restoration unit 184 may skip the 3Drestoration processing. Alternatively, the 3D restoration unit 184 mayexecute the 3D restoration processing by using only right images at alltimes. When the image IMGn is a left image, the 3D restoration unit 184may skip the 3D restoration processing.

The 3D restoration unit 184 may switch images used in the 3D restorationprocessing in accordance with the type of the image IMGn. In otherwords, the 3D restoration unit 184 may execute the 3D restorationprocessing by using only left images and then may execute the 3Drestoration processing by using only right images. The 3D restorationunit 184 generates a first 3D shape in the 3D restoration processingthat uses only left images. The 3D restoration unit 184 generates asecond 3D shape in the 3D restoration processing that uses only rightimages. When the consistency between the first 3D shape and the second3D shape is secured, the 3D restoration unit 184 may integrate the first3D shape and the second 3D shape. For example, when Step S153 isexecuted, the consistency between the first 3D shape and the second 3Dshape is secured.

It is predicted that the optical-path-setting unit 55 has difficulty inswitching imaging optical paths by following a first pattern describedbelow in terms of heat generation and the durability of the shutter whenthe endoscope device 1 is actually used. The first pattern indicates thetype of images to be generated. According to the first pattern, imagesare switched for each frame, and a left image b and a right image I_(R)are alternately generated.

-   -   A first pattern: I_(L), I_(R), I_(L), I_(R), . . .

Therefore, it is assumed that the optical-path-setting unit 55 switchesimaging optical paths by following a second pattern described below. Thesecond pattern indicates the type of images to be generated. Accordingto the second pattern, images are switched only at a specific timing.

-   -   A second pattern: I_(L), I_(L), I_(L), . . . , I_(L), I_(R),        I_(L), I_(R), I_(L), I_(L), I_(L), . . .

For example, a user inputs a switching instruction to switch imagingoptical paths into the endoscope device 1 by operating the operationunit 4 or a touch panel. When the user has input the switchinginstruction into the endoscope device 1, the control unit 180 causes theoptical-path-setting unit 55 to switch imaging optical paths.Alternatively, the control unit 180 detects movement of the distal end20 by using only a left or right image. When the movement of the distalend 20 has not been detected, the control unit 180 causes theoptical-path-setting unit 55 to switch imaging optical paths.

When the user has input the switching instruction into the endoscopedevice 1 or the control unit 180 has not detected the movement of thedistal end 20, imaging optical paths are switched. On the other hand,when the user has not input the switching instruction into the endoscopedevice 1 or the control unit 180 has detected the movement of the distalend 20, the imaging device 28 generates only a left or right image. In acase in which the second pattern is used, Step S154 is basicallyexecuted. In such a case, Step S153 is executed, for example, at atiming at which the user inputs the switching instruction into theendoscope device 1.

The 3D restoration unit 184 may use the first 3D shape and the second 3Dshape described above to enhance the accuracy of a 3D shape by followingthe epipolar constraint at a timing at which Step S153 is executed.Alternatively, the 3D restoration unit 184 may use the first 3D shapeand the second 3D shape described above to finalize or update the scaleof a subject by following the epipolar constraint at a timing at whichStep S153 is executed.

Details of the 3D restoration processing in the fourth embodiment willbe described by using FIG. 38 . Left images IL1 to IL8 and right imagesIR1 and IR2 are generated at different time points. These imagesconstitute stereo images. Time passes from the left to the right in FIG.38 .

The 3D restoration unit 184 executes the 3D restoration processing andgenerates 3D data DT1, 3D data DT2, and 3D data DT3. Points included inthe 3D data and images used to generate the points are associated bylines. For example, a left image IL1 and a left image IL2 are used togenerate a point PT10. A point PT11 is seen only in the left image IL1,and 3D data at the point PT11 are lost.

It is assumed that the distal end 20 moves between a first timing and asecond timing and moves between the second timing and a third timing.The imaging device 28 generates a left image IL3 at the first timing,generates a right image IR1 at the second timing, and generates a leftimage IL4 at the third timing.

It is assumed that the distal end 20 does not move between the thirdtiming and a fourth timing and does not move between the fourth timingand a fifth timing. The imaging device 28 generates the left image IL4at the third timing, generates a right image IR2 at the fourth timing,and generates a left image IL5 at the fifth timing.

While the left images IL1 to IL3 are generated, the imaging optical pathis set to the first optical path L1. The imaging device 28 does notgenerate a right image that constitutes a stereo image with the leftimage IL1, IL2, or IL3. Therefore, the 3D restoration unit 184 executesStep S154.

After the left image IL3 is generated, the imaging optical path is setto the second optical path L2 and the imaging device 28 generates aright image IR1. The distal end 20 moves between a timing at which theimaging device 28 generates the left image IL3 and a timing at which theimaging device 28 generates the right image IR1. In this case, the rightimage IR1 is determined to be inappropriate for the 3D restorationprocessing and is not used in the 3D restoration processing. Processingof determining whether the right image IR1 is appropriate for the 3Drestoration processing is not shown in FIG. 37 .

After the right image IR1 is generated, the imaging optical path is setto the first optical path L1 and the imaging device 28 generates theleft image IL4. The 3D restoration unit 184 executes Step S152 anddetermines whether the distal end 20 is stationary. The 3D restorationunit 184 determines that the distal end 20 is moving and executes StepS154. At this time, data of the right end in the 3D data DT1 shown inFIG. 38 and data of the left end in the 3D data DT2 shown in FIG. 38 areconnected.

After the left image IL4 is generated, the imaging optical path is setto the second optical path L2 and the imaging device 28 generates aright image IR2. The distal end 20 does not move between a timing atwhich the imaging device 28 generates the left image IL4 and a timing atwhich the imaging device 28 generates the right image IR2. In this case,the 3D restoration unit 184 executes Step S153. The 3D restorationprocessing that uses the epipolar constraint is executed, and the scaleof the already generated 3D data is uniquely determined.

After the right image IR2 is generated, the imaging optical path is setto the first optical path L1 and the imaging device 28 generates theleft image IL5. The 3D restoration unit 184 executes Step S152 anddetermines whether the distal end 20 is stationary. The 3D restorationunit 184 determines that the distal end 20 is not moving and executesStep S153.

Thereafter, left images IL6 to IL8 are generated. The imaging device 28does not generate a right image that constitutes a stereo image with theleft image IL6, IL7, or IL8. Therefore, the 3D restoration unit 184executes Step S154. At this time, the 3D data DT2 and the 3D data DT3shown in FIG. 38 are connected.

In the fourth embodiment, the endoscope device 1 acquires a left imageand a right image in a time-division manner by using the stereo opticaladaptor 30 and uses the left image and the right image in the 3Drestoration processing. Therefore, the endoscope device 1 can stablyexecute the 3D restoration processing with high accuracy, compared tothe first embodiment in which a single-eye optical adaptor is used.

While preferred embodiments of the invention have been described andshown above, it should be understood that these are examples of theinvention and are not to be considered as limiting. Additions,omissions, substitutions, and other modifications can be made withoutdeparting from the spirit or scope of the present invention.Accordingly, the invention is not to be considered as being limited bythe foregoing description, and is only limited by the scope of theappended claims.

What is claimed is:
 1. A three-dimensional data generation method of generating three-dimensional data indicating a three-dimensional shape inside a turbine by using a processor, the method comprising: acquiring two or more images of a component inside the turbine, wherein the component includes a first object capable of moving inside the turbine and a second object that is stationary inside the turbine, wherein the two or more images are generated by an imaging apparatus including a tubular insertion unit that acquires an optical image inside the turbine, wherein the insertion unit is inserted into the turbine through a hole formed in the turbine, wherein a moving direction of the insertion unit when the insertion unit is inserted into the turbine is different from a moving direction of the first object, and wherein a relative position of the first object to the insertion unit is different between timings at which the imaging apparatus generates images while the first object moves; detecting two or more correspondence regions that are the same regions of the component in at least two images included in the two or more images; determining whether at least part of a region of each of the two or more images is a change region or a non-change region, wherein the change region is a region of the component of which coordinates in an image generated by the imaging apparatus change, and wherein the non-change region is a region of the component of which coordinates in an image generated by the imaging apparatus do not change; and generating the three-dimensional data by using a correspondence region determined to be the change region among the two or more correspondence regions without using a correspondence region determined to be the non-change region among the two or more correspondence regions.
 2. The three-dimensional data generation method according to claim 1, wherein, after the two or more correspondence regions are detected, determination as to whether the at least part of the region is the change region or the non-change region is executed.
 3. The three-dimensional data generation method according to claim 1, wherein, after determination as to whether the at least part of the region is the change region or the non-change region is executed, the two or more correspondence regions are detected by using the change region without using the non-change region.
 4. The three-dimensional data generation method according to claim 1, wherein the first object includes a rotor blade, and wherein the second object includes a stator blade or a shroud.
 5. The three-dimensional data generation method according to claim 1, wherein part of the first object is concealed by the second object in the two or more images.
 6. The three-dimensional data generation method according to claim 5, wherein the second object includes an object that conceals part of the first object in the two or more images, and wherein the second object includes an object, part of which is concealed by the first object in the two or more images.
 7. The three-dimensional data generation method according to claim 1, wherein the imaging apparatus generates the two or more images at two or more different timings, and wherein determination as to whether the at least part of the region is the change region or the non-change region is executed based on a moving amount of each of the correspondence regions between at least two images included in the two or more images.
 8. The three-dimensional data generation method according to claim 1, wherein determination as to whether the at least part of the region is the change region or the non-change region is executed based on a difference of a pixel value between at least two images included in the two or more images.
 9. The three-dimensional data generation method according to claim 1, wherein determination as to whether the at least part of the region is the change region or the non-change region is executed by determining a subject seen in one image included in the two or more images.
 10. The three-dimensional data generation method according to claim 1, further comprising: determining a position of the insertion unit inside the turbine; and executing processing of notifying a user of information indicating a change of the position of the insertion unit when the position has changed.
 11. The three-dimensional data generation method according to claim 1, further comprising: calculating an area of the non-change region in an image included in the two or more images before generating the three-dimensional data; and executing processing of notifying a user of an alert when the area is larger than a predetermined value.
 12. The three-dimensional data generation method according to claim 1, wherein the first object rotates inside the turbine due to a driving force generated by a driving device, and wherein the method further comprises: determining whether the first object has rotated once inside the turbine; and executing processing of notifying a user of information indicating that the first object has rotated once inside the turbine when it is determined that the first object has rotated once inside the turbine.
 13. The three-dimensional data generation method according to claim 1, wherein the first object rotates inside the turbine due to a driving force generated by a driving device, wherein detection of the two or more correspondence regions, determination as to whether the at least part of the region is the change region or the non-change region, and generation of the three-dimensional data are repeatedly executed while the first object rotates, and wherein the generation of the three-dimensional data is stopped and the detection of the two or more correspondence regions and the determination as to whether the at least part of the region is the change region or the non-change region are continued when the rotation of the first object has been stopped.
 14. The three-dimensional data generation method according to claim 13, wherein the generation of the three-dimensional data is restarted when the first object starts to rotate again.
 15. The three-dimensional data generation method according to claim 13, further comprising determining a rotation state of the first object by using an image included in the two or more images.
 16. The three-dimensional data generation method according to claim 13, further comprising determining a rotation state of the first object by monitoring a state of the driving device.
 17. The three-dimensional data generation method according to claim 1, wherein the insertion unit is fixed inside the turbine.
 18. The three-dimensional data generation method according to claim 1, wherein the imaging apparatus is a borescope.
 19. A three-dimensional data generation system that generates three-dimensional data indicating a three-dimensional shape inside a turbine, the system comprising: an imaging apparatus including a tubular insertion unit that acquires an optical image inside the turbine, wherein the imaging apparatus is configured to generate two or more images of a component inside the turbine, wherein the component includes a first object capable of moving inside the turbine and a second object that is stationary inside the turbine, wherein the insertion unit is inserted into the turbine through a hole formed in the turbine, wherein a moving direction of the insertion unit when the insertion unit is inserted into the turbine is different from a moving direction of the first object, and wherein a relative position of the first object to the insertion unit is different between timings at which the imaging apparatus generates images while the first object moves; and a three-dimensional data generation device including a processor configured to: acquire the two or more images; detect two or more correspondence regions that are the same regions of the component in at least two images included in the two or more images; determine whether at least part of a region of each of the two or more images is a change region or a non-change region, wherein the change region is a region of the component of which coordinates in an image generated by the imaging apparatus change, and wherein the non-change region is a region of the component of which coordinates in an image generated by the imaging apparatus do not change; and generate the three-dimensional data by using a correspondence region determined to be the change region among the two or more correspondence regions without using a correspondence region determined to be the non-change region among the two or more correspondence regions.
 20. The three-dimensional data generation system according to claim 19, wherein the imaging apparatus and the three-dimensional data generation device are included in an endoscope device.
 21. The three-dimensional data generation system according to claim 19, wherein the imaging apparatus is included in an endoscope device, and wherein the three-dimensional data generation device is included in an external device other than the endoscope device.
 22. A non-transitory computer-readable recording medium storing a program causing a computer to execute processing of generating three-dimensional data indicating a three-dimensional shape inside a turbine, the processing comprising: acquiring two or more images of a component inside the turbine, wherein the component includes a first object capable of moving inside the turbine and a second object that is stationary inside the turbine, wherein the two or more images are generated by an imaging apparatus including a tubular insertion unit that acquires an optical image inside the turbine, wherein the insertion unit is inserted into the turbine through a hole formed in the turbine, wherein a moving direction of the insertion unit when the insertion unit is inserted into the turbine is different from a moving direction of the first object, and wherein a relative position of the first object to the insertion unit is different between timings at which the imaging apparatus generates images while the first object moves; detecting two or more correspondence regions that are the same regions of the component in at least two images included in the two or more images; determining whether at least part of a region of each of the two or more images is a change region or a non-change region, wherein the change region is a region of the component of which coordinates in an image generated by the imaging apparatus change, and wherein the non-change region is a region of the component of which coordinates in an image generated by the imaging apparatus do not change; and generating the three-dimensional data by using a correspondence region determined to be the change region among the two or more correspondence regions without using a correspondence region determined to be the non-change region among the two or more correspondence regions. 